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I.  INTRODUCTION 


A.  Background 

In  March  1982,  the  HELBAT  (Human  Engineering  Laboratory  Battalion  Artillery  Test) 
Executive  Committee  agreed  that  the  Ballistic  Research  Laboratory  Artillery  Control 
Environment  (ACE)  and  HELBAT  activities  should  be  combined  to  develop  a  Command  Post 
Exercise  Research  Facility  (CPXRF).  The  CPXRF  technology  will  primarily  be  used  for 
research,  development,  testing  and  evaluation  (RDTfcE)  work  in  automatic  data  processing 
(ADP)  fire  support  control  systems  using  commercial  ADP  technology;  a  secondary  usage  is  the 
training  of  the  tactical  ADP  operators  under  controlled  conditions.  Further,  an  ACE/CPXRF 
Subcommittee  was  formed  to  provide  joint  DARCOM-TRADOC  guidance  in  the  development  of 
ACE  technology  and  use  of  the  CPXRF.  The  ACE  software  is  a  key  tool  in  the  CPXRF.  The 
software  features  the  ability  to  automatically  load  live  players  with  messages  produced  by  target 
acquisition  and  fire  direction  simulators  while  recording  all  the  message  traffic  that  flows 
between  the  live  and  simulated  players. 

An  overview  of  the  CPX  Research  Facility  and  ACE  program  is  given  in  the  1982  Sept-Oct 
issue  of  the  Field  Artillery  Journal  in  an  article  "HELBAT/ACE  Fire  Support  Control  Research 
Facility”  by  Mr.  Barry  Reichard.  The  layout  of  the  facility  is  shown  in  Figure  2. 


B.  Purpose 

The  experiment  detailed  in  this  report  was  the  first  test  in  which  military  players  were 
interfaced  with  the  Artillery  Control  Environment  (ACE)  software.  The  purpose  of  this 
experiment  was  to  demonstrate  the  feasibility  of  using  the  automated  techniques  of  the  CPX 
Research  Facility  for  fire  support  control  experiments. 

To  demonstrate  this  capability,  a  study  of  the  effects  of  message  intensity  and 
communication  degradation  on  the  Fire  Support  Team  Headquarter’s  (FIST  HQ)  ability  to 
perform  fire  support  coordination,  with  a  newly  developed  FIST  Digital  Message  Device  (DMD), 
was  performed.  Message  intensity  was  defined  to  be  a  function  of  message  ^ype,  message  rate, 
and  message  content. 


11.  TEST  CONCEPT 


A.  Objectives 

1.  To  determine  the  effect  of  message  intensity  on  the  FIST  HQ  ability  to  perform  fire 
support  coordination. 

2.  To  determine  the  effect  of  communication  degradation  on  the  FIST  HQ  ability  to 
perform  fire  support  coordination. 
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Figure  2.  Camand  Post  E;<ercise  Research  Facility. 


3.  To  determine  if  message  intensity  and  degraded  communication  have  a  combined  effect 
on  fire  support  coordination. 


B.  Measures  of  Performance 

A  measure  of  performance  (MOP)  is  a  response  that  is  used  to  quantify  the  effects  of  the 
factors  to  be  evaluated.  Because  all  of  our  objectives  investigated  the  effect  on  fire  support 
coordination,  the  measures  of  performance  were  the  same  for  all  three  objectives.  The  following 
measures  of  performance  were  computed  for  each  two  hour  cell  of  the  test: 

1.  Number  of  messages  serviced  by  the  FIST  HQ.  This  number  provides  information  on 
the  message  traflSc  at  the  FIST  HQ  under  the  different  conditions  and  can  be  translated  into  net 
usage. 


2.  Frequency  count  by  number  of  transmissions  for  messages  acknowledged.  The  FIST 
DMD  has  a  one  character  field  for  try  number  that  cycles  modulo  4  (i.e.,  0,1, 2, 3, 0,1 ,2,3,0,...).  It 
was  noticed  in  HELBAT  8  data,  that  more  than  four  transmissions  were  sometimes  necessary  to 
get  an  acknowledgement  back  on  a  message.  TACFIRE  uses  the  try  number  in  the  FIST  DMD 
message  to  determine  what  authenticator  to  select  for  comparison  to  the  DMD  message. 
Therefore,  if  the  number  of  transmissions  exceeds  four,  the  FIST  DMD  displays  a  message  to 
the  operator  to  contact  his  destination  by  voice  to  synchronize  authenticator  codes.  This  voice- 
digital  contention  then  causes  more  problems  for  a  net  that  is  already  experiencing 
communications  problems. 

3.  Frequency  count  of  the  number  of  receptions  of  a  given  message  at  the  FIST  HQ. 
Ciiven  that  a  message  is  transmitted  more  than  once,  the  message  can  also  be  received  more 
than  once.  Many  times  a  message  is  sent  and  received,  but  the  acknowledgement  is  deleted  by 
communication  degradation.  The  message  is  then  retransmitted  and  perhaps  received  again. 

4.  Service  time  distribution,  where  service  time  is  defined  to  be  the  time  required  for  the 
FIST  HQ  to  service  a  message  starting  from  the  time  the  acknowledgement  (,^CK)  is  sent  from 
the  FIST  DMD  acknowledging  receipt  of  a  message  to  the  time  the  response  message  is  first 
transmitted.  This  measure  includes  the  time  a  message  spends  in  the  FIST  DMD  message  queue 
plus  the  processing  and  decision  time  of  the  FIST  HQ. 

5.  Manual  transmission  time  distribution,  where  manual  transmission  time  is  defined  to  be 
the  time  from  first  transmission  of  the  response  message  by  the  operator  to  the  time  an 
acknowledgement  is  received  for  that  message.  The  FIST  HQ  have  completed  the  decision 
making  at  this  poit.t,  but  must  continue  to  send  the  message  until  an  acknowledgement  is 
received.  In  degraded  communications  this  time  may  not  be  inconsequential.  Also,  the  FIST 
HQ  cannot  process  other  messages  while  transmitting  manually. 

6.  Number  of  fire  missions  completed/number  of  fire  missions  initiated.  The  FIST  HQ 
was  given  two  hours  and  ten  minutes  to  complete  two  hours  of  scenario.  A  complete  fire 
mission,  by  definition,  is  a  call  for  fire  (FR  GRID),  a  message-to-observer  (MTO),  at  least  one 
SHOT  and  an  end-of-mission  (EOM) 


7.  Number  of  fire  missions  completed/number  of  fire  missions  expected.  The  number  of 
fire  missions  expected  is  the  number  of  fire  missions  in  the  scenario  database.  This  was  to 
measure  if  the  FIST  HQ  could  complete  all  fire  missions  in  the  two-hour  scenario  database 
within  the  two  hours  and  ten  minutes  allotted. 


C.  Scope 

The  FIST  HQ  was  a  four-man  team  consisting  of: 

1.  the  fire  support  team  chief 

2.  the  fire  support  sergeant 

3.  two  radio  telephone  operator /drivers. 

The  FIST  chief  was  available  to  the  FIST  HQ  for  initial  supervision  only.  As  per  typical 
operating  procedures,  the  FIST  chief  may  be  absent  for  extended  periods  of  time  (hypothetically 
accompanying  the  company  commander). 

The  FIST  HQ  was  task-loaded  by  software  interactively  simulating  three  platoon-level 
forward  ob'crvers.  The  software  FOSCE  (Forward  Observer  SCEnario)  used  tactical  scenarios 
developed  by  Mr.  Arthur  Long  of  the  US  Army  Field  Artillery  Board.  This  scenario  or  input 
database  is  detailed  in  Section  III-D,  "Input  Database.” 

The  FIST  HQ  had  direct  access  to  fire  support  from  a  company-level  mortar  platoon  fire 
direction  center  (FDC)  and  a  generic  field  artillery  fire  direction  center.  All  FDC  operations 
were  simulated  interactively  by  software.  The  FIST  HQ  determined  the  proper  action  (based  on 
the  FIST  chief’s  guidance  and  training)  for  each  fire  request:  either  to  deny  the  request,  service 
the  request  with  mortars  or  forward  the  request.  Fire  support  was  unlimited,  that  is,  not 
constrained  by  ammunition  resupply;  this  was  not  to  be  a  study  factor  for  this  experiment. 

.Ml  members  of  the  FIST  HQ  crew  were  trained  in  the  operation  of  the  FIST  DMD  to  give 
the  FIST  chief  flexibility  in  managing  his  team. 

P.  Limitations 

1.  .Ml  forward  observer  addresses  were  placed  in  the  review  mode  in  the  FIST  DMD 
subscriber  table. 

2.  .After  deciding  a  fire  request  should  be  handled  either  by  the  mortars  or  forwarded  to 
the  FDC,  tlie  fire  mission  was  forwarded  in  the  automatic  mission  mode.  That  is.  all 
subsetiueut  messages  for  that  fire  mission  were  automatically  routed  through  the  FIST  DMD. 
Operator  intervention  was  needed  only  if  a  message  did  not  get  acknowledged  in  four 
transmissions.  He  was  then  notified  that  the  message  did  not  get  ACKed;  the  message  was 
placed  in  his  message  queue  and  transmitted  manually. 
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3.  No  FIST  HQ  initiated  missions  were  included. 

4.  No  tactical  chores  were  performed,  e.g.,  guard  duty,  close  station  march  order, 
emplacement,  etc. 

5.  All  communications  were  digital;  the  simulators  could  not  respond  to  voice 
communications. 


E.  Test  Configuration 

Figure  3  shows  the  nodes  that  were  played  in  the  first  military  player  test;  (1)  the  FIST 
HQ  equipped  with  the  FIST  DMD  in  the  mock-up  vehicle  interacting  through  Ether,  the 
intracomputer  communications  network,  with  the  three  forward  observer  scenario  programs,  (2) 
the  mortar  fire  direction  simulator  and  battalion  fire  direction  simulator.  Figure  4  shows  how 
these  components  were  netted. 
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III.  RESOURCE  REQUIREMENTS 


A.  Software 

ACE  software  permitted  real-time  fire  support  command  and  control  functions  to  be 
exercised  in  a  controlled  laboratory  environment.  The  software  is  written  in  the  C  programming 
language  and  is  designed  to  run  under  the  4.1bsd  (Berkeley)  UNIX  operating  system. 

The  two  simulator  programs,  FOSCE  (Forward  Observer  SCEnario)  and  FDS/MFDS  (Fire 
Direction  Simulator/Mortar  FDS)  interactively  simulated  both  tactical  equipment  and  its 
human  operators.  FOSCE  mimicked  the  actions  of  the  platoon  forward  observers  that  work  for 
the  FIST  HQ  while  the  FDS  programs  simulated  generic  artillery  battalion  and  mortar  fire 
direction  centers,  respectively,  executing  fire  missions.  It  bad  to  be  determined  exactly  how  the 
simulators  should  react  to  the  many  different  events  that  could  occur  during  the  scenario.  This 
required  group  participation  and  each  possible  event  had  to  be  discussed.  Many  different  factors 
were  taken  into  consideration  such  as  tactical  realism,  physical  constraints,  and  the  test  design. 
Most  of  the  events  could  be  handled  in  more  than  one  way  and  a  decision  had  to  be  made  (that 
was  sometimes  arbitrary)  as  to  how  the  simulator  should  react.  This  problem  was  compounded 
by  the  introduction  of  degraded  communications.  Even  a  simple  adjust  fire  mission  includes 
over  30  messages  that  could  be  randomly  deleted  during  degraded  communications  operations, 
and  the  simulators  had  to  adjust  to  react  to  this  (note  Figure  5). 

Some  operations  that  are  normally  handled  via  voice  with  TACFIRE  had  to  be 
implemented  using  digital  meant  for  the  simulators.  For  example,  fire  missions  from  FOSCE 
could  be  rejected  or  ended  by  sending  a  freetext  message  "MISSION  REJECTED."  If  a  target 
number  had  been  assigned,  FOSCE  would  send  an  End  Of  Mission  and  Surveillance  (ES) 
message  and  then  wait  for  the  next  mission  An  ES  message  would  cause  the  FDS  or  MFDS  to 
end  a  mission  already  in  progress. 
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1ST  Headquarters  Experiment. 


ETHER:  Intra-computer  eommunlcstlons  net 
FOSCE:  Forward  Observer  SCEnario  program 
FOS:  Fire  Direction  Simulator  program 


Figure  4.  Test  Configuration. 


Figure  5.  Simple  Adjust  Fire  Mission. 


Despite  the  capability  built  into  the  simulators  to  handle  degraded  communications, 
"deadlock”  situations  still  occurred.  This  would  happen  when  a  message  that  was  lost  from  one 
simulator  was  required  by  another  simulator  to  continue  processing.  A  common  example  of  this 
occurred  when  a  Subsequent  Adjust  message  from  FOSCE,  which  was  to  be  relayed 
automatically  to  the  FDS  by  the  FIST  DMD,  was  lost  during  the  relay.  Although  the  FIST 
DMD  warned  the  operator  when  such  a  situation  existed,  the  warning  was  sometimes  missed, 
inadvertently  deleted,  or  more  urgent  problems  forced  a  delay  in  reacting  to  the  warning  and  it 
was  forgotten.  Hence,  FOSCE  was  content  because  the  message  it  sent  was  acknowledged  by 
the  P'IST  DMD;  FOSCE  was  waiting  for  a  SHOT  message.  But  the  Subsequent  Adjust  message 
for  which  the  FDS  was  waiting  never  arrived;  both  simulators  were  waiting  for  something  from 
the  other.  It  was  noted  by  a  Field  Artillery  School  instructor  that  this  situation  is  not 
uncommon  in  the  real  world.  To  solve  this  problem,  the  simulators  had  to  be  able  to  respond 
to  queries  from  the  FIST  HQ.  Freetext  fire  mission  "status”  messages  were  defined  so  that  the 
FIST  HQ  could  inquire  about  the  current  status  of  a  fire  mission  by  referencing  either  a  Target 
Number  or  an  Observer  Identification  number  and  DMD  mission  buffer.  These  inquiries  could 
be  sent  to  either  the  FOSCE  or  FDS  programs,  and  based  upon  the  response,  the  FIST  HQ 
could  take  action  to  fix  the  deadlock  situation  or  end  the  mission. 

Special  messages  were  developed  to  allow  the  simulators  to  add  information  to  the 
database.  These  messages  were  sent  to  reserved  addresses  so  that  they  could  be  easily  identified 
in  the  database  later.  The  simulators  did  not  expect  acknowledgement  for  these  messages. 
Some  of  these  messages  were  displayed  by  ADIS  for  the  controllers  to  identify  special  events, 
such  a«  a  simulator  not  receiving  an  acknowledgement  after  four  transmission  attempts,  or  the 
reception  of  a  duplicate  fire  mission  by  the  FDS  (i.e.,  same  as  one  already  in  progress).  Other 
messages  were  simply  entered  into  the  database  for  use  during  data  reduction.  These  were 
typically  messages  that  announced  that  a  particular  simulator  was  beginning  or  ending  a  fire 
mission.  They  identified  the  time  that  the  simulators  believed  a  fire  mission  started  or  ended 
regardless  of  the  actions  of  the  live  players.  These  messages  were  used  by  the  data  reduction 
software  to  identify  when  events  actually  began  or  ended  despite  the  confusion  caused  by  the 
cokom  unications  degradation. 

I'fae  capability  to  start  or  end  a  cell  anywhere  within  that  cell  was  implemented  to  allow 
the  controller  to  restart  a  cell  if  a  computer  hardware  malfunction  was  encountered,  thus 
eliminating  the  need  to  redo  a  complete  cell.  This  capability  was  utilized  only  two  or  three 
times  during  the  entire  experiment  but  it  did  save  several  hours  of  test  time. 

The  major  components  of  the  ACE  software  are  described  in  the  following  sections. 

1.  Ether.  Ether  is  a  single  program  which  functions  as  an  intra-computer 
communications  network.  Computer  ports  are  assigned  to  communication  nets.  Ether  accepts 
a  message  from  a  port  and  transmits  it  to  all  other  ports  on  the  assigned  net.  Message  collisions 
are  prevented  by  separately  buffering  each  message  within  Ether.  Although  Ether  prevents 
collisions  between  messages,  the  TACFIRE  digital  network  is  a  half-duplex  system;  only  one 
player  at  a  time  can  transmit  a  message  without  a  collision.  Thus,  messages  between  the  FIST 
DMD  and  the  Bit  Boxes  bad  a  small  probability  of  collision.  This  non-zero  probability  was 
observed  and  measured  during  the  experiment. 


Each  net  is  assigned  a  probability  of  message  loss  which  ranges  from  zero  to  one.  If  the 
probability  of  message  loss  is  zero,  the  net  is  an  ideal  net,  and  all  messages  are  sent  to  each  port 
on  the  net.  If  the  probability  of  message  loss  is  greater  than  zero,  a  uniform  random  number 
generator  is  used  to  decide  whether  or  not  a  message  is  lost.  Lost  messages  are  not  transmitted 
to  any  port  on  the  net.  Acknowledgements  (ACKs)  are  treated  the  same  as  any  other  message. 

Ether  maintains  a  log  file  of  each  message  which  it  receives.  In  addition  to  the  raw 
message,  the  log  contains  the  times  (Julian  day,  hour,  minute,  second)  for  the  start  of  the 
message,  the  end  of  the  preamble  and  the  end  of  the  message. 

2.  ACE  Display  and  Information  System  fADlSI.  A  real  lime  display  program,  named 
ADIS  (ACE  Display  and  Information  System),  allowed  the  controllers  to  see  all  the  messages  as 
they  passed  between  real  and  simulated  players.  The  display,  along  with  a  chronological  listing 
of  the  scenario,  was  used  by  the  controllers  to  track  the  progress  in  the  test  cells.  It  proved 
essential  in  identifying  and  collecting  information  on  unusual  events,  which  normally  developed 
quickly  and  were  hard  to  trace.  This  was  even  more  important  during  the  extensive  debugging 
phase  of  the  experiment.  The  Field  Artillery  School  instructor  also  found  it  helpful  in  following 
the  progress  of  the  students  during  the  training  phase. 

A  camera  and  microphone  were  placed  in  the  FIST  vehicle  mock-up  to  record  the  face  of 
the  FIST  DMD  and  the  crew’s  conversations.  These  were  simultaneously  recorded  with  the 
ADIS  Display  so  that  a  complete  picture  could  be  obtained  at  a  later  time  to  identify  specific 
parts  of  the  scenario  that  caused  problems  and  the  events  that  led  up  to  them.  When  the  ADIS 
display  was  combined  with  the  actual  actions  of  the  crew,  it  presented  a  comprehensive  picture 
of  how  the  test  group  (FIST)  interacted  with  the  entire  system. 

ADIS  utilizes  a  CRT  (cathode  ray  tube)  terminal  to  display  in  real  time  the  messages  being 
transmitted  through  Ether.  The  terminal  screen  is  divided  into  eight  columns  which  are  labeled 
for  the  players  (see  Figure  0).  Each  message  is  displayed  as  two  lines  in  both  the  sender's  and 
receiver’s  columns.  The  message  first  appears  in  the  sender’s  column.  The  first  line  contains  the 
message  type  and  target  number  if  it  has  been  assigned.  The  second  character  in  the  second 
line  is  a  indicating  ’’sender,”  and  the  time  sent  is  given.  The  message  will  then  appear  in 
the  "receiver’s”  column.  The  first  line  is  the  same  as  in  the  "sender’s”;  the  second  character  in 
the  second  line  gives  the  address  of  the  "senderf  and  the  time  received  is  displayed.  When  the 
acknowledgement  is  sent  by  the  "receiver?  an  is  displayed  as  the  first  character  in  the 
second  line  of  the  "receiver?  and  when  the  acknowledgement  is  received  by  the  "sender?  an 
is  displayed  as  the  first  character  in  the  second  line  of  the  "sender"  message.  If  the  message 
is  deleted  by  Ether,  "MSG  LOST”  appears  in  the  receiver’s  column.  Below  the  columns,  the 
last  message  sent  is  fully  depicted,  time  tagged,  and  deciphered.  At  the  bottom  of  the  screen, 
the  time  from  the  start  of  the  run  is  displayed. 

3.  Forward  Observer  Scenario  (FOSCEI.  The  forward  observer  scenario  program  reads  a 
database  of  forward  observer  (FO)  messages  and  transmits  the  messages  as  if  they  were  being 
generated  by  a  real  FO  with  a  DMD.  Each  message  is  time-tagged  in  the  datab.ise  and  sent  by 
FOSCE  at  the  appropriate  time.  FOSCE  waits  10  seconds  for  an  ACK;  if  one  is  not  received,  it 
retransmits  the  message.  Three  retransmissions  (for  a  total  of  four)  occur  before  the  program 
gives  up.  If  no  ACK  is  received  after  four  transmissions  of  a  fire  request,  FOSCE  waits  four 
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minutes  (hopefully  long  enough  to  resynchronite  nuthenticator  codes  with  TACFIRE  by  voice) 
and  then  recycles  the  call-for-fire.  FOSCE,  after  receiving  an  ACK  on  a  request  for  fire,  waits  10 
minutes  for  a  message-to-observer  (MTO)  message.  If  one  is  not  received,  it  resends  the  request 
for  fire.  After  receipt  of  the  MTO,  FOSCE  waits  up  to  5  minutes  for  a  SHOT  message.  Once 
the  SHOT  message  is  received,  FOSCE  waits  at  least  90  seconds  before  transmitting  subsequent 
adjust  (SA)  messages.  Because  no  voice  communication  was  allowed,  FOSCE  was  made  smart 
enough  to  respond  to  freetext  messages  asking  for  the  status  of  a  particular  fire  mission  by 
target  number  or  for  the  status  of  FOSCE  itself,  that  is,  active  or  not  active. 

4.  Fire  Direction  Simulator  IFDSI.  The  fire  direction  simulator  consists  of  four  programs 
which  perform  a  limited  number  of  TACFIRE/BCS  functions.  FDS  accepts  fire  request 
messages,  prioritizes  them,  assigns  target  numbers  and  generates  MTO  and  SHOT  messages. 
Times  between  receipt  of  fire  requests  and  transminion  of  MTO  and  SHOT  messages  vary  due 
to  the  computer  workload  and  an  internal  random  number  generator.  If  the  computer  is  not 
overloaded,  the  time  between  the  fire  request  and  the  MTO  randomly  varies  between  15  and  45 
seconds,  and  the  time  between  the  SA  and  the  SHOT  varies  between  20  and  40  seconds.  The 
number  of  simultaneous  missions  which  the  FDS  will  process  may  be  specified.  If  the  number  of 
missions  exceeds  the  maximum,  the  FDS  will  process  missions  based  on  mission  priority.  During 
this  experiment,  the  FDS  could  handle  up  to  10  missions  simultaneously,  which  by  intent  was 
not  a  limitation  on  the  system.  The  FDS  could  be  queried  by  the  FIST  HQ  as  to  the  status  of  a 
particular  fire  mission  by  target  number  or  by  observer  identification  number  and  mission 
buffer. 

5.  Mortar  Fire  Direction  Simulator  (MFDSI.  The  mortar  FDS  simulates  the  digital 
communication  of  the  company  mortar  FDC.  It  is  a  special  version  of  the  FDS  program  which 
will  only  accept  one  fire  mission  at  a  time. 

6.  Bit  Box  Program  (BBPI.  The  Bit  Box  interface  program  accepts  messages  from  Ether 
and  transmits  them  to  a  computer  port  which  is  connected  to  a  Bit  Box.  The  program  also 
reacb  messages  from  the  computer  port  and  transmits  them  to  Ether. 

B.  Hardware 

1.  Two  Bit  Boxes.  Bit  Boxes  are  microprocessor-based  MODEMs  which  enable 
TACFIRE  hardware  to  interface  with  commercial  computers.  Bit  Boxes  accept  TACPIRE 
messages  from  wire  line  or  radio,  perform  error  correction  and  convert  the  messages  to  RS232 
ASCII  characters,  which  commercial  computers  can  accept.  They  also  accept  messages  from  the 
computer,  add  the  error  correction  bits,  time  disperse  the  messages  and  transmit  them  over  wire 
line  or  radio  in  TACFIRE  format  (FSK). 

2.  FIST  DMD.  The  FIST  digital  message  device  that  was  used  in  the  experiment  was 
one  of  four  experimental  design  models  (EDM  #2)  that  are  in  existence.  It  was  a  prototype 
model,  and  not  a  production  model. 

3.  VAX  11/750  Computer.  The  VAX  11/750  computer  was  dedicated  to  running  the 
experiment  and  had  no  other  processes  running  during  the  test.  The  operating  system  was  the 
4.1bsd  (Berkeley)  UNIX. 


Test  participants  were  collectively  trained  at  the  Human  Engineering  Laboratory  in  the 
operation  of  the  FIST  DMD  by  CPT  Gahagan,  an  instructor  from  the  Gunnery  Department  of 
the  US  Army  Field  Artillery  School.  The  Human  Engineering  Laboratory  provided  training 
equipment  for  the  students.  The  test  participants  were  trained  Fire  Support  Teams  (MOS  13F) 
from  the  82nd  Airborne  Division,  Ft.  Bragg. 

The  FIST  could  continue  fire  support  coordination  operations  through  30  percent 
communications  degradation  once  trained  to  do  so.  A  key  lesson  they  learned  was  to  use  a 
"wait  and  see”  technique  after  failing  to  get  an  ACK  to  a  message  after  four  tries.  Usually  it  is 
the  ACK,  rather  than  the  message,  that  is  lost.  (This  situation  could  become  quite  common 
when  a  mobile  observer  with  a  low  power  radio  is  communicating  with  a  station  that  has  a  good 
antenna  and  a  high  powered  radio,  for  example,  an  FDC.)  Hence,  by  waiting  a  few  minutes  the 
expected  response  message  was  received  even  though  the  ACK  never  was.  The  FIST  also 
learned  to  use  the  "status”  messages  to  find  and  fix  deadlock  situations  thus  resorting  to  digital, 
rather  than  voice,  means  to  correct  problems.  However,  they  still  depended  upon  paper  and 
pencil  to  keep  track  of  what  target  numbers  were  active  along  with  the  progress  of  each  mission. 


D.  Input  Database 

The  tactical  scenario  database  contained  fire  support  control  messages  for  a  limited 
scenario  of  a  mechanized  infantry  battalion  of  an  armored  division.  The  SCORES,  Europe  111. 
Sequence  2.^  was  used  to  generate  fire  missions  expected  to  be  fired  by  a  field  artillery  battalion 
in  sustained  combat  operation.  The  battalion  is  constrained  by  ammunition  resupply  under 
normal  operations;  however,  it  was  decided  that  ammunition  resupply  should  not  be  a  limiting 
condition  in  this  test.  The  entire  scenario  was  played  in  retrograde  mode. 

Scenario  definition  is  still  very  subjective.  For  this  experiment  a  scenario  was  defined  to  be 
a  time  ordered  list  of  digital  messages,  and  in  this  case,  messages  that  would  be  received  by  the 
FIST  HQ  from  its  platoon  forward  observers  (FO)  using  TACFIRE  Digital  Message  Devices 
(DMD).  It  was  surprising  that  no  definition  of  "intensity"  could  be  found  that  was  given  in 
terms  of  number  of  fire  missions  (FM)  or  messages  per  hour.  Hence,  a  "reasonable"  guess  had 
to  be  made:  Low  Intensity  -  1  Fire  Mission  per  FO  per  hour.  Medium  -  2  FM/FO/hour,  and 
high  -  3  FM/FO/hour.  The  number  of  Artillery  Target  Intelligence  (ATI)  messages  was  varied 
inversely  from  the  FMs,  and  an  Immediate  Smoke  mission  was  added  to  each  medium  and  high 
intensity  cell  (each  cell  was  2  hours  long). 

Because  cells  of  the  same  intensity  were  to  be  compared,  several  other  criteria  were 
imposed  on  the  scenario  to  insure  that  task  loading  on  the  FIST  HQ  didn’t  vary  significantly 
between  cells  of  the  same  intensity.  The  ratio  of  Fire  For  Effect  (FFE)  to  Adjust  Fire  (AF) 
missions  was  chosen  as  2:1  (as  per  Ft.  Sill’s  direction),  the  number  of  adjustments  in  each  AF 
mission  was  chosen  as  three,  and  one  fire  mission  in  each  2-hour  cell  was  designated  as  urgent 
rather  than  normal  priority.  After  the  scenario  was  received,  it  was  realized  that  the  time 
interval  between  fire  missions  was  also  a  significant  factor  that  influences  the  loading  on  the 
FIST  HQ.  Since  this  timing  wasn't  specified  in  the  scenario  definition,  all  the  fire  mission  time- 


tags  were  changed  manually  so  that  the  intervals  between  the  fire  missions  were  the  same  for 
each  cell  of  the  same  intensity.  This  was  a  time  consuming  procedure  (60  man-hours).  Some 
factors  that  were  not  analyzed  in  the  first  CPX  experiment  (but  perhaps  should  have  been)  were 
the  ranges  to  the  targets,  the  target  descriptions,  and  how  often  or  which  targets  were  in  range 
of  the  available  fire  support  assets.  These  factors  would  certainly  affect  the  FIST  HQ 
perception  of  the  threat  and  perhaps  the  urgency  attached  to  tasks,  both  of  which  might 
influence  the  results  of  tests  like  this  one. 

The  database  consisted  of  36  two-hour  cells  of  messages:  12  two-hour  cells  of  low  intensity, 
12  two-hour  cells  of  medium  intensity  and  12  two-hour  cells  of  high  intensity.  Intensity  is 
defined  by  the  number  and  type  of  initiating  messages  per  two-hour  cell  as  given  in  Figure  7, 
and  the  message  stream  that  follows  each  initiating  message  as  given  in  Figure  8.  It  can  be  seen 
that  intensity  is  a  function  of  the  number  of  initiating  messages  and  their  subsequent  messages. 
The  36  two- hour  cells  of  data  were  arranged  such  that  all  permutations  of  the  three  intensities 
(L-M-H)  appeared  twice.  Ninety  percent  of  the  fire  missions  had  normal  priority,  and  the  other 
ten  percent  had  urgent  priority. 

The  process  of  verifying  that  the  large,  144-hour  scenario  contained  what  was  requested 
could  only  be  achieved  practicably  through  automatic  means.  The  scenario  was  delivered  on 
computer  tape  and  loaded  into  the  BRL  computeifs)  where  it  could  be  manipulated  with  the 
many  standard  software  packages  included  with  the  UNIX  Operating  System.  Several  other 
programs  were  written  to  examine  the  database  and  display  information  concerning  the  factors 
listed  above.  (Most  of  these  programs  were  written  in  a  convenient  pattern  scanning  and 
processing  language  named  AWK.) 

The  messages  in  the  scenario  had  to  be  converted  from  a  "pseudo  TACFIRE”  format  to 
the  "Fixed  Format"  used  by  DMDs.  The  program  written  to  do  this  was  "table  driven.”  This 
type  of  program  is  relatively  quick  to  write  and  fast  in  execution,  however,  it  is  intolerant  to 
errors;  therefore,  any  deviation  from  the  expected  input  format  produced  an  error.  There  were 
many  format  errors  in  the  input  database  (e.g.,  the  abbreviation  for  a  target  disposition, 
DISPO,  was  often  missing  the  "17  DSPO)  that  required  manual  alterations,  an  unexpected  time 
consuming  task  on  the  large  database. 

IV.  DATA  COLLECTION 


A.  Experimental  Design 

1.  Factors.  The  two  factors  that  were  tested  in  this  experiment  were  message  intensity 
and  communication  degradation.  Three  levels  of  message  intensity  were  tested  with  each  of 
three  levels  of  communication  degradation  giving  nine  test  combinat'ons.  The  levels  of  each 
factor  are  abbreviated  as  follows: 


FACTORS 


i 

E  1)  INTENSITY  (per  two  hour  block) 


MESSAGE  TYPE 

LEVELS 

Low 

Medium 

High 

Fire  Mission  1,  Fire  For  Effect 

4 

8 

12 

Fire  Mission  2,  Adjust  Fire 

2 

4 

6 

Fire  Mission  3,  Immediate  Smoke 

0 

1 

1 

Artillery  Target  Intelligence 

18 

12 

6 

1 


>  2)  COMMUNICATION  DEGRADATION 

00%  Message  Loss 

►  15%  Message  Loss 

30%  Message  Loss 

*  Figure  7.  Factors  for  FIST  Experiment. 
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INTENSITY 


MESSAGE  SEQUENCE 


1)  Ariillcry  Target  Tntdligcnce 
ATI  FO— *FIST— ♦FDC 


18  12  6 


2)  Fire  Mission,  Fire  for  Effect 
FR  GRID  F0~»  FISr-4FDS 
MTO  FO«-FlSr«--FDS 

SHOT  FO^-'FICT*' FDS 

EOM  FO  “♦FIST— »FDS 


3)  Fire  Mission,  Adjust  Fire 

FRGRID 

FO-^FIST— ♦FDS 

MTO 

FO«-FISr4— FDS 

SHOT 

FO^- FIST*— FDS 

SA(1) 

FO -►FlSr— *FDS 

SHOT 

FO*-FISrr*-FDS 

SA(2) 

FO-»FISr— ♦FDS 

SHOT 

F0*-Fisrr*-FDS 

SA(3) 

FO  — ►HSr— ►FDS 

SHOT 

FO*-HST*-FDS 

EOM 

FO-»FlST— ►FDS 

4)  Fire  Mission,  Immed.  Smoke 
Same  as  Adjust  Fire  Misaon 


Figure  8.  Subsequent  Message  Flow  (w/o  ACKs)  by  Message  Type 


Message  Intensity 


L  =  low 
M  =  medium 
H  =  high 

C’ommunications  Degradation 
0  =  0^  degradation 

1  ==  degradation 

2  =  30^  degradation 


2.  Design  Matrix.  It  was  decided  that  the  shortest  reasonable  time  to  test  any  one  of  the 
nine  treatment  combinations  was  two  hours.  Since  the  testing  of  all  nine  treatment 
combinations  required  a  minimum  of  18  hours,  which  realistically  could  not  be  completed  in  one 
day,  a  randomized  incomplete  block  design  was  constructed  so  that  the  day-to-day  variability 
would  not  influence  the  results.  The  nine  treatment  combinations  were  divided  into  blocks  of 
three,  and  the  three  blocks  were  run  over  a  three  day  period.  The  assignment  of  the  treatment 
combinations  into  blocks  was  based  on  a  confounding  scheme.  This  scheme  a.ssured  that  the 
effects  of  message  intensity  (I),  communication  degradation  (C)  and  the  interaction  of  these  two 
factors  (1  X  C)  on  a  FIST  HQ  ability  to  perform  fire-support  coordination  could  be  measured. 
Because  time  constraints  permitted  only  two  replications,  part  of  the  precision  of  the  estimate  of 
the  interaction  was  sacrificed  (i.e.,  blocks  within  replicate  1  were  confounded  with  the  linear 
component  of  the  I  x  C  interaction  and  blocks  within  replicate  2  were  confounded  with  the 
quadratic  component  of  the  I  x  C  interaction).  Randomization  of  treatment  combinations 
within  blocks  and  blocks  within  days  was  performed. 

The  experiment  was  repeatec  for  four  FIST  teams,  so  that  team-to-team  variability  was 
included.  In  addition,  software  changes  were  implemented  between  teams  2  and  3  as  a  result  of 
information  from  a  pilot  test.  One  significant  change  was  to  have  the  FDS  send  one  SHOT 
message  per  call-for-fire  rather  thau  one  SHOT  message  per  volley.  Capability  for  status 
requests  was  implemented  in  the  FDS  at  this  time  also.  Because  of  these  changes,  software  was 
made  a  factor  in  the  experiment  so  that  the  variability  due  to  the  software  changes  could  be 
detected. 

The  design  matrix  is  showf  ia  Figure  9.  The  FIST  teams  were  tested  sequentially,  one  at 
a  time,  for  six  days.  The  six  days  are  shown  in  the  design  matrix  and  the  tests  were  run  in  the 
order  given  within  each  day. 
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Figure  9.  Design  Matrix  for  Experiment. 
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V.  DATA  ANALYSIS 


A.  Statistical  Analysis 

Reducing  45,000  messages  recorded  in  this  72  two-hour  cell  experiment  into  meaningful 
information  was  not  a  simple  task.  The  reaction  of  the  FIST  DMD  operators  to  degraded 
communications  often  produced  results  that  were  unexpected  or  difficult  to  trace.  As  mentioned 
earlier,  the  slight  delay  in  acknowledgement  messages  (ACK)  coupled  with  the  heavy  message 
traffic  often  made  it  difficult  to  match  ACKs  to  messages  after  the  fact.  Human  analysis  was 
still  required  to  retrieve  many  results;  however,  many  of  the  standard  programs  available  with 
the  UNIX  Operating  System  were  invaluable  to  assist  in  this  work. 

1.  Kffect  of  Factors  on  Message  Traffic.  The  total  number  of  messages  generated  for 
each  experimental  condition  over  a  two>hour  cell  was  used  t '  evaluate  and  validate  the  effect 
that  the  different  factors  and  their  interactions  had  on  message  traffic.  Based  on  the  way 
intensity  and  communication  degradation  were  defined  in  planning  this  experiment,  we  expected 
these  two  factors  to  have  a  significant  effect  on  message  traffic.  An  increa.se  in  intensity  level 
resulted  in  an  increase  in  the  number  of  messages  generated.  Similarly,  an  increase  in 
communication  degradation  resulted  in  an  increase  in  the  number  of  messages  it  took  to 
complete  a  fire  mission  or  to  forward  an  artillery  target  intelligence  message.  To  some  this  may 
seem  counter  intuitive;  however,  in  degraded  communications,  some  messages  were  being  sent 
but  not  received,  and  this  resulted  in  retransmissions  which  increased  message  traffic.  The 
other  factors  specified  in  the  design,  including  the  two  different  Fire  Direction  Simulator 
software  programs,  were  also  included  in  this  anidysis. 

The  number  of  messages  observed  in  each  test  cell  are  shown  in  Figure  10.  \-^  analysis  of 
variance  was  performed  on  this  data  with  all  replicate  interaction  terms  poo  ed  fur  the  error 
term.  A  second  analysis  of  variance  procedure  was  then  performed  with  additional  interaction 
terms  found  not  to  be  significant  also  being  pooled  with  error.  The  ANOVA  table  for  the  final 
reduced  model  is  shown  in  Table  1.  It  should  be  noted  that  since  block  was  confounded  with 
components  of  the  intensity-degradation  interaction,  it  was  not  meaningful  to  test  any  term  in 
the  model  containing  block.  A  double  star  next  to  the  F-statistic  indicates  that  the  factor  is 
significant.  Based  on  the  calculated  F-values,  intensity,  degradation,  intensity-degradation 
interaction,  software,  and  intensity-software  interaction,  were  found  to  have  a  significant  effect 
on  the  message  traffic. 

The  effect  that  intensity,  degradation  and  their  interaction  have  on  message  traffic  is 
summarized  in  Table  2.  Table  2  gives  the  average  number  of  messages  per  two-hour  cell,  /i,  and 
the  number  of  cells  in  the  average,  N,  for  the  given  factors  and  their  marginal  effects  (averages 
over  the  rows  and  columns).  Looking  at  the  average  number  of  messages  generated  for  each 
level  of  intensity  presented  in  the  right-hand  column  of  Table  2,  one  sees  that  there  was  a 
significant  increase  from  361.46  to  882.50  as  intensity  increased.  Similarly,  an  increase  in 
communication  degradation  increased  the  average  message  traffic  flow  from  462.13  to  798.58.  In 
addition,  in  comparing  the  mean  change  between  the  different  levels  of  communication 
degradation  for  each  level  of  intensity,  a  positive  interaction  effect  can  be  noted.  There  was  an 
increase  in  the  mean  of  about  200  messages  between  0  percent  and  30  percent  degradation  for 
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TABLE  2.  Number  of  Cells  in  the  Average  (N)  and 
Average  Number  of  Messages  Per  Two  Hour  Cell  ( fi) 


Intensity 

Communication  Degradation  (%) 

00 

15 

30 

LOW 

8 

266.88 

8 

351.88 

8 

465.63 

Hj 

MEDIUM 

8 

461.00 

8 

636.00 

8 

798.38 

HIGH 

8 

658.50 

8 

857.25 

8 

1131.75 

24 

882.50 

24 

462.13 

24 

615.04 

24 

798.58 

N 

low  intensity  compared  to  an  increase  of  over  300  messages  for  medium  and  500  messages  for 
high  intensity. 

The  effect  that  software  and  the  software>intensity  interaction  had  on  message  traffic  is 
summerizod  in  Table  3.  The  average  number  of  messages  generated  per  two- hour  block  for  the 
original  FDS  software  program  was  704.67  compared  to  545.83  for  the  modified  program.  The 
software  was  changed  to  produce  a  shot  message  for  every  call  for  fire  instead  of  every  volley 
which  is  a  more  realistic  representation  of  how  TACFIRE/BCS  functions.  Therefore,  one  would 
expect  the  average  message  flow  to  be  less  for  software  2  than  1.  Also,  one  would  expect  a 
greater  change  between  low,  medium  and  high  intensity  for  software  1  than  2.  From  Table  3, 
the  (lifierence  between  means  for  low  and  high  intensity  for  software  1  is  over  600  messages 
compared  to  a  difference  of  slightly  over  400  for  the  modified  software.  To  obtain  a  realistic 
description  of  the  effect  that  message  intensity  and  communication  degradation  had  on  network 
message  traffic  flow  and  on  the  Fire  Support  Teams’  ability  to  perform  effective  fire  support 
coordination,  the  analysis  from  this  stage  on  was  based  on  the  second  half  of  the  experiment 
using  the  modified  software. 


TABLE  3.  Number  of  Two  Hour  Cells  in  the  Average  (N) 
and  Average  Number  of  Messages  Per  Two  Hour  Cell  (ft) 
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The  number  of  messages  per  unit  time  can  be  translated  into  net  usage.  Net  usage  is 
defined  to  be  the  percent  of  time  a  net  is  occupied  by  message  transmissions.  Message  traffic 
during  this  test  was  handled  by  two  nets.  Net  1  represented  the  company  fire  control  net  and 
consisted  of  the  links  between  the  three  forward  observer  simulators,  the  FIST  HQ  and  the 
company  mortar  fire  direction  simulator.  Net  2  was  the  fire  direction  (FD)  net  and  connected 
the  Fire  Support  Team  and  the  battalion  fire  direction  center  simulator  (Ft)S),  In  a  direct 
support  artillery  battalion,  there  are  three  fire  direction  nets,  one  for  each  battery,  that  each 
link  TACFIRE  (bn  FDC),  a  BCS  (battery  FDC),  a  VFMED  (bn  FSE)  and  three  FIST  DMDs 
(FIST  HQ).  The  fire  direction  net  was  not  fully  loaded  during  this  experiment;  only  one  Fire 
Support  Team  exercised  the  FDS.  Also,  the  FIST  HQ  have  the  capability  to  monitor  two  other 
nets,  the  company  command  net  and  the  battalion  mortar  fire  direction  net  which  were  not 
used  during  the  test. 

The  time  a  n.t  is  busy  is  a  function  of  the  number  of  messages  passing  through  the 
network,  the  lengths  of  the  messages  and  the  appropriate  preamble  times  as  imposed  by  the  Bit 
Box,  the  DMD,  or  the  FDS  software.  FOSCE  had  no  preamble  set  during  the  test. 

Acknowledgemevits  and  48  character  DMD  fixed  format  messages  were  used  to  compute  net 
usage.  Neither  of  the  message  types  were  compressed  at  any  time  during  the  test.  Figure  11 
illustrates  the  format  of  a  DMD  message  transmission  which  always  includes  a  preamble,  synch 
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characters,  and  the  message.  Preambles,  which  take  a  certain  amount  of  time  to  be 
transmitted,  allow  the  transmitting  and  receiving  radios  to  reach  operating  conditions  and  help 
synchronize  the  DMD  receiving  devices.  The  synch  characters  serve  as  a  start  of  message 
indicator  and  tell  whether  or  not  the  incoming  message  is  encrypted.  The  message  is  comprised 
of  three  parts:  the  header,  which  identifies  the  message  transmitter  and  assures  that  the 
message  is  valid,  the  body  of  the  message,  and  the  tail,  indicating  the  end  of  message 
transmission  and  which,  by  TACFIRE  protocol,  must  contain  at  least  4  EOTs.  All  messages, 
including  acknowledgements,  were  passed  through  a  Bit  Box  which  imposed  a  preamble  time  of 
0.1  seconds.  In  addition,  the  DMD  and  the  FDS  software  each  imposed  their  own  preamble  of 
1.0  seconds  when  transmitting. 
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Figure  11.  Message  Transmission  Format. 


The  32-bit  synch  characterization  is  the  same  for  acknowledgements  and  other  message 
types;  however,  the  average  length  of  the  message  itself  varies  between  the  two.  The  message 
length  of  an  acknowledgement  is  one  block  long.  A  block  consists  of  16  characters,  and  a 
character  is  12  bits,  therefore,  the  message  length  of  an  acknowledgement  is  192  bits.  Including 
the  synch  characters,  the  total  number  of  message  transmission  bits  for  acknowledgements 
becomes  224.  All  48  character  DMD  messages  are  three  blocks  long  resulting  in  a  message 
length  of  576  bits  and  a  total  message  transmission  length  of  608  bits.  All  message 
transmissions  were  sent  at  a  rate  of  1200  bits  per  second. 

Net  usage  was  computed  by  dividing  the  total  message  transmission  time,  including 
preamble  time,  on  a  net  by  the  total  net  time  available.  Table  4  gives  the  average  net  usage  in 
percent  for  each  intensity-degradation  combination  and  the  marginal  net  usage  for  each  of  these 
factors.  It  can  be  seen  that  although  a  change  in  degradation  level  from  0  to  15  percent 
increased  the  average  number  of  messages  by  33  percent  (from  462  to  615),  the  increase  in  net 
usage  was  only  between  one  and  two  percent.  Similarly,  an  increase  from  0  to  30  percent 
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degradation  produced  an  increase  of  73  percent  (from  462  to  799)  in  the  average  number  of 
messages;  the  net  usage  increased  between  2  and  4  percent.  The  net  usage  would  increase 
considerably  if  the  system  were  fully  loaded  and  voice  communication  added.  The  percentages 
indicate  that  the  actual  transmission  time  for  a  large  number  of  digital  messages  is  very  small 
and  that  little  reduction  in  total  fire  mission  time  could  be  gained  by  decreasing  the 
transmission  time  of  digital  messages.  However,  it  is  important  to  note  that  the  FD  net  (Net  2) 
usage  ranges  from  2.6  to  10.0%  with  only  one  of  the  three  or  four  FISTs  operating  and  without 
the  message  traffic  between  TACFIRE,  BCS  and  VFMEDs,  which  are  long  variable  format 
messages;  therefore,  net  contention  could  be  a  serious  problem  when  the  net  is  fully  loaded. 

2.  Freouenev  Count  bv  Number  of  Transmissions  of  Messages  Acknowle 


Theoretically,  the  number  of  transmissions  it  takes  for  a  message  to  successfully  reach  its 
destination  and  for  an  acknowledgement  to  be  received  by  the  sender  should  only  be  affected  by 
the  percent  of  communication  degradation  in  the  communication  networks.  Given  the  actual 
percent  degradation,  the  theoretical  distribution  of  bow  many  times  a  message  is  sent  before  it 
is  acknowledged  can  be  determined  for  each  level  of  communication  degradation.  When  there  is 
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DO  communication  degradation,  all  messages  should  be  acknowledged  on  the  first  try.  In 
15  percent  degradation  the  probability  that  a  message  gets  through  and  is  acknowledged  on  ;iny 
try  is  ( I-.15){1-.15)=.7225.  The  probability  that  a  message  does  not  get  acknowledged  on  a 
given  try  is  1-.7225.  Using  these  probabilities,  the  probability  that  a  message  is  acknowledged 
in  a  given  number  of  transmissions  can  be  computed.  Table  5  gives  the  distributions  of  the 
theoretical  probability  of  getting  a  message  acknowledged  in  n  transmissions  for  15  and  30 
percent  degradation. 


TABLE  5.  Theoretical  Probability  That  A  Message  Is 
Acknowledged  In  n  Transmissions 
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Using  the  theoretical  probabilities  from  above  and  the  total  number  of  messages  actually 
acknowledged  under  each  degradation  level,  the  actual  effect  of  ACE’s  communication 
degradation  can  be  checked.  Figure  12  shows  the  distribution  of  total  messages  acknowledged 
by  try  number  in  "perfect”  communications  (0  percent  degradation)  for  software  2.  "Perfect" 
communication  was  not  quite  perfect. 

Figures  13  and  14  give  the  same  distributions  for  15  and  30  percent  degradation.  Very 
good  agreement  was  observed,  and  when  tested  statistically  (See  Appendix  A),  the  number  of 
messages  acknowledged  by  number  of  transmissions  was  a  function  of  communication 
degradation  only  and  was  not  influenced  by  intensity,  team  variability  or  learning. 
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Distribution  of  Messages  Acknowledged  by  Try  Number  for  0%  Degradation 


The  Bit  Boxes  (MODEMs  that  convert  between  Frequency  Shift  Keyed  (FSK)  TACFIRE 
signals  and  standard  RS-232  signals)  do  not  prevent  collisions  from  occurring;  therefore,  evr  ti 
"perfect"  communications,  wasn’t.  The  FIST  DMD  contained  a  "net  monitoring"  function  to 
reduce  message  collisions.  This  feature  was  not  implemented  in  the  Bit  Bo.xes,  and  it  was 
originally  believed  that  the  lack  of  this  was  the  reason  for  most  of  the  collisions;  however,  this 
was  not  the  case.  Of  the  4764  messages  that  were  sent  during  the  second  half  of  the  experiment 
under  "perfect”  communications  conditions,  109  had  to  be  sent  more  than  once.  There  was  only 
one  collision  resulting  from  two  original  messages  being  transmitted  simultaneously;  the  rest 
were  collisions  between  a  message  and  an  acknowledgement  (ACK).  A  look  at  these  messages 
revealed  that  3  out  of  4  of  them  were  sent  from  the  FIST  DMD  (the  rest  were  sent  from  the 
simulators).  This  exposed  a  basic  problem,  not  in  the  hardware,  but  in  the  ACE  software.  The 
actual  T.^CFIRE  hardware  transmits  ACKs  immediately;  however,  those  in  the  experiment 
normally  took  2  to  3.5  seconds.  This  acknowledgment  delay  was  more  than  enough  to  trigger 
the  FIST  DMD  to  retransmit  the  message.  Hence,  3  out  of  4  of  these  message  collisions  would 
not  have  happened  in  the  "real  world”  or  only  about  25  of  4700  messages  would  have  actually 
collided  if  the  FOs  were  not  listening  to  the  net  before  transmitting.  Whether  the  delay  in  the 
ACK  was  caused  by  the  ACE  Ether  program  or  the  simulators  is  not  known;  however,  a  new 
Ether  program  is  being  developed  that  wifi  solve  this  problem,  and  the  simulators  can  be 
modified  to  send  acknowledgements  faster. 

3.  Frequency  Count  of  the  Number  of  Receptions  of  a  Given  Message.  As  mentioned  in 
a  previous  section  many  retransmissions  of  messages  occurred  in  degraded  communications. 
Retransmissions  were  necessary  either  because  the  message  itself  or  its  associated 
acknowledgement  was  deleted  due  to  intentional  communications  degradation.  If  the  message 
gets  to  its  destination  and  the  acknowledgment  is  deleted,  duplicate  copies  of  a  given  messac 
can  be  received.  The  distributions  of  the  number  of  times  a  given  message  was  received  at  the 
FIST  ilQ  for  15  and  30  percent  degradation  are  shown  in  Figures  15  and  16,  respectively.  In 
15-perccnt  degradation,  17  percent  of  the  messages  received  by  the  FIST  HQ  were  duplicate 
receptions.  In  30-percent  degradation,  31  percent  of  the  messages  received  were  duplicate 
receptions.  The  FIST  DMD  does  no  checking  for  duplicate  receptions  of  a  given  message, 
therefore,  the  burden  was  on  the  FIST  HQ  crew  to  recognize  a  message  as  a  duplicate  and 
delete  it  from  the  queue.  The  FIST  HQ  usually  recognized  duplicate  messages,  but  occasionally 
would  service  a  duplicate  reception  of  a  fire  request  or  ATI  message.  If  more  than  one  copy  of  a 
given  fire  request  was  received  by  the  FDS  it  would  inform  the  FIST  HQ  that  it  was  a  duplicate 
target  and  not  initiate  another  fire  mission.  Once,  however,  a  FIST  HQ  sent  one  copy  of  a  fire 
reque.st  to  the  FDS  and  a  second  reception  of  the  fire  request  to  the  MFDS;  both  missions  were 
run  to  completion. 

Duplicate  receptions  cause  serious  problems  when  conducting  a  mission  in  the  automatic 
mission  mode.  The  FIST  HQ  crew  do  not  process  any  messages  except  the  original  fire  request 
messages  in  the  autom;  ic  mission  mode.  Therefore,  if  the  FIST  DMD  receives  duplicate  copies 
of  non- fire-request  messages,  they  are  all  forwarded.  Furthermore,  if  the  FIST  DMD  receives, 
say,  two  copies  of  a  message  from  the  FDS,  they  may  in  turn  generate  four  copies  of  that  same 
message  at  the  FO  because  of  retransmissions  by  the  FIST  DMD.  In  a  worst  case,  each  of  the 
four  automatic  transmissions  of  a  message  by  TACFIRE  could  be  received  by  the  FIST  DMD 
and  the  FIST  DMD  could  generate  four  copies  of  each  auton atically,  meaning  that  the  FO 
could  possibly  (although  not  probably)  receive  sixteen  copies  of  t  given  message  if  no 
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Figure  15.  Distribution  of  Messages  by  Times  Received  for  lb  Degradation. 
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acknowlrdgcments  are  received  aud  the  automatic  mission  mode  is  being  used. 

4.  Time  Required  to  Service  a  Message.  This  section  investigates  the  effect  that 
degradation  and  intensity  had  on  the  time  it  took  for  FIST  HQ  3  and  4  to  service  fire  request 
(FR)  messages  and  artillery  target  intelligence  (ATI)  messages.  Only  those  teams  tested  under 
the  second  set  of  software  were  investigated  here  since  software  was  a  signiBcant  factor  and 
since  the  second  set  of  software  was  more  realistic  tactically.  Since  6re  requests  are  given  a 
higher  priority  than  ATls  and  require  more  processing  by  the  FIST  HQ  crew,  message  type  had 
to  be  considered  a  factor  in  this  analysis. 

As  the  data  was  checked  for  completeness,  it  was  noted  that  the  distribution  of  service 
time  was  skewed  and  that  the  variance  of  the  observations  under  various  experimental 
conditions  exhibited  discrepancies.  A  check  for  homogeneity  of  variance  using  Bartlett's  test 
confirmed  the  latter  observation.  In  addition,  several  experimental  groups  had  observations 
that  were  extremely  large  (over  four  standard  deviations  from  the  group  mean)  and  atypical  of 
the  majority  of  the  service  times  observed  under  the  same  experimental  conditions.  These 
observations  comprised  slightly  more  than  four  percent  of  the  total  service  limes  observed. 
They  were  removed  from  the  analysis  of  variance  procedure  found  below,  but  were  considered  in 
interpreting  the  final  results  below.  The  median  for  each  experimental  condition  is  given  in 
Table  6. 

Further  investigation  of  the  data  revealed  a  positive  correlation  between  the  standard 
deviations  and  the  experimental  group  means.  Correlation  between  the  standard  deviations  and 
group  means  is  often  accompanied  by  marked  non-normality  and  non-homogeneity  of  variance, 
and  indicates  that  the  particular  form  of  the  original  observations  is  unsuitable  for  ANOVA 
procedures.  However,  a  transformation  can  be  determined  which  makes  the  standard  deviation 
independent  of  the  mean,  corrects  non-homogeneity  and  also  results  in  the  observations  being 
distributed  more  normally.  In  general,  if  a  significant  functional  relationship  between  the 
standard  deviation  and  the  group  means  can  be  determined,  then  the  transformation  is  the 
integral  of  the  reciprocal  of  this  functional  relationship.  Following  this  procedure,  the  following 
transformation  was  developed: 

1.3  In  (  -  2.6  -f  .8  (  service  time  )) 

The  transformed  data  became  more  normal  and  the  assumption  of  homogeneity  of  variance  was 
confirmed. 

.\n  analysis  of  variance  pro.edure  was  then  performed  on  the  transformed  data.  One 
slight  modification  to  thi.-  procedure  was  that  due  to  unequal  experimental  group  sires,  the  sum 
of  squares  for  all  terms  in  the  model,  except  the  error  term,  was  weighted  by  the  harmonic 
mean  I'lie  final  reduced  ANOVA  table  is  presented  in  Table  7. 

The  most  significant  term  in  the  analysis  was  ream.  The  median  service  time  for  team  3 
was  14. .a  seconds  which  is  substantially  higher  (73  percent)  when  compared  to  the  8.5  seconds 
for  team  4.  This  trend  is  prevalent  for  both  fire  requests  and  ATI  messages,  but  is  magnified 
when  one  considers  ju.st  fire  requests.  As  suspected,  type  of  message  also  influenced  service 


TABLE  6.  Median  Service  Time 
by  Experimental  Condition 
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time.  Although  fire  requests  have  a  higher  priority  than  ATIs,  they  contain  more  information 
that  has  to  be  recorded  and  verified  by  the  FIST  HQ.  Therefore,  it  was  not  surprising  that  the 
median  time  (13.5  seconds)  for  fire  requests  was  55  percent  higher  than  the  median  service  time 
(8.5  seconds)  for  ATIs. 


TABLET.  Analysis  of  Variance 
(Effect  on  Service  Time) 


SOURCE 

DEGREES  OF 
FREEDOM 

SUM  OF 
SQUARES 

MEAN 

SQUARE 

F 

RATIO 

Replication 

1 

5.62 

5.62 

8.64** 

Message  Type 

1 

6.0  i 

6.01 

9.25** 

Block  within  Rep 

4 

16.45 

4.11 

Message  Type  X 
Block  within  Rep 

4 

7.36 

1.84 

Team 

1 

150.98 

150.98 

232.3** 

Team  X 

Block  within  Rep 

4 

17.1 

4.28 

Intensity 

2 

14.77 

7.38 

11.2** 

Intensity  X 
Message  Type 

2 

9.48 

4.74 

7.18** 

Intensity  X 
Team 

2 

7.25 

3.63 

5.5** 

Degradation 

2 

52.68 

26.34 

39.91** 

Degradation  X 
Message  Type 

2 

5.52 

i 

2.76 

4.18** 

Degradation 

X  Team 

2 

2.60 

1.30 

Intensity  X 
Degradation 

4 

31.68 

7.92 

12.01** 

Intensity  X 
Degradation 

X  Team 

4 

11.99 

3.00 

4.54‘* 

Pooled  Error 

790 

520.9 

.66 

Total 

825 

860.39 

The  ANOVA  table  revealed  that  communications  degradation  and  intensity  had  a 
signihcant  effect  on  the  FIST  HQ  service  time.  An  increase  in  intensity  or  degradation  resulted 
in  an  increase  in  the  time  it  took  for  the  FIST  HQ  to  service  a  message.  Because  message  type 
was  also  significant,  the  effects  of  intensity  and  degradation  for  each  message  type  were 
considered  separately.  The  median  service  time  for  ATIs  increased  12  percent  from  low  to  high 
intensity  as  observed  in  the  right  marginal  of  Table  8.  However,  the  FIST  HQ  ability  to  service 
fire  requests  remained  essentially  the  same  in  either  low  or  high  intensity  as  shown  in  Table  8. 
One  possible  explanation  is  that  as  intensity  increased,  a  larger  proportion  of  the  total  effort 
was  used  to  ser\’ice  the  fire  request  messages  because  they  were  higher  priority  then  ATIs. 
Consequently,  ATIs  were  not  serviced  as  quickly. 

The  effect  that  degradation  had  on  service  time  is  consistent  with  the  above  trend  for  both 
ATIs  and  fire  requests.  As  observed  in  examining  the  bottom  marginals  of  Table  8,  an  increase 
in  degradation  from  0  to  30-percent  resulted  in  the  FIST  HQ  median  service  time  increasing  29 
percent  and  i3  percent  for  fire  requests  and  ATIs,  respectively. 

TABLE  8.  Intensity  by  Degradation 
Median  Service  Time 
for  Fire  Requests  (FRs) 

and  Artillery  Target  Intelligence  (ATIs)  Messages 
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Figure  17.  Median  Service  Time  for  Low  Intensity. 


Figure  19.  Median  Service  Time  for  High  Intensity. 


For  ATIs,  the  median  service  time  increased  only  slightly  as  degradation  increased  from  0 
to  30  percent  for  low  or  medium  intensity  as  shown  in  Table  8  and  Figures  17  and  18. 
Similarly,  for  fire  request  messages,  the  service  time  increase  from  0  to  30  percent  degradation 
was  only  4  percent  in  medium  intensity.  This  trend  was  more  noticeable  in  low  intensity  where 
the  median  FIST  HQ  service  time  for  fire  requests  increased  almost  28  percent  as  degradation 
increased  from  0  to  30  percent.  However,  in  high  intensity,  the  increase  from  0  to  30  percent 
degradation  resulted  in  a  substantial  increase  in  service  time  for  both  ATIs  and  fire  request 
messages  when  compared  to  any  increase  observed  in  low  or  medium  intensity.  The  median 
service  time  for  fire  requests  increased  46  percent  from  0  to  30  percent  degradation  and  for  .^Tls 
increased  179  percent.  This  was  due  to  the  fact  that  the  largest  median  service  time  observed 
for  ATIs  and  fire  requests  occurred  under  30  percent  degradation  and  high  intensity.  In 
addition,  it  was  only  under  this  condition  that  the  median  service  time  (19.5  seconds)  for  .\Tls 
was  higher  than  the  median  service  time  (17.5  seconds)  for  fire  requests,  as  depicted  in  Figure 
19.  This  seems  to  substantiate  the  hypothesis  that  under  increased  workload,  the  FIST  HQ 
spends  more  time  trying  to  service  fire  request  messages  while  ATIs  are  left  in  the  DMD  queue. 
It  also  may  mean  the  point  was  reached  at  which  fire  support  coordination  operations  begins  to 
seriously  degrade. 

Although  replication  (learning)  was  significant,  only  a  slight  decrease  (8  percent)  in  service 
time  was  observed  between  replicate  1  and  replicate  2. 

The  final  step  in  this  analysis  was  to  categorize  the  removed  data  (outliers)  by  various 
experimental  conditions.  The  following  trends  were  worth  noting.  Of  the  36  service  times 
removed  from  the  data  base,  over  one  third  were  observed  under  30  percent  degradation  and 
high  intensity.  In  addition,  75  percent  were  observed  from  30  percent  degradation  with  over  92 
percent  coming  from  two-hour  cells  that  were  run  under  15  or  30  percent  degradation.  These 
observations  substantiate  the  hypothesis  that  increased  degradation  and  the  combined  effect  of 
30  percent  degradation  and  high  intensity  causes  unpredictable  delays  for  the  FIST  HQ  in 
servicing  messages. 

5.  Manual  Transmission  Time  Distribution.  Manual  transmission  time  is  defined  to  be 
the  time  between  the  first  transmission  of  a  message  by  the  FIST  DMD  operator  until  an 
acknowledgement  is  received  for  that  message.  Manual  transmission  time  was  computed  using 
only  fire  request  and  artillery  target  intelligence  message  types  since  these  were  the  only  message 
types  (excluding  freetext)  which  were  not  automatically  forwarded  by  the  FIST  DMD  in 
automatic  mission  mode.  Transmission  time  for  both  of  these  initiating  message  types  was 
found  to  be  greatly  affected  by  the  percent  of  communication  degradation  in  the  communication 
networks.  This  is  consistent  with  results  found  for  the  frequency  count  of  messages  by  number 
of  transmis.sions;  that  is,  the  more  transmissions  required  until  an  acknowledgement  was 
received,  the  longer  the  transmission  time.  The  time  it  took  for  a  message  to  successfully  reach 
its  destination  and  for  an  acknowledgement  to  be  received  by  the  FIST  increased  with  an 
increase  in  degradation.  This  trend  is  evident  in  Figures  20  thru  22  which  show  the  distribution 
of  manual  transmission  times  less  than  180  seconds  for  0,  15  and  30-percent  degradation, 
respectively.  All  messages  in  0  percent  degradation  took  less  than  180  seconds  and  are  shown 
on  Figure  19.  Two  transmission  times,  one  at  251  and  the  other  at  273  seconds  are  not  shown 
on  the  distribution  for  15  percent  degradation,  but  are  reflected  in  the  statistics  computed. 
Nine  service  times  are  not  shown  in  30  percent  degradation.  They  occur  at  186,  209,  216,  222, 
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Figure  20.  Distribution  of  Transmission  Times  for  0%  Degradation. 
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Distribution  of  Transmission  Times  for  15%  Degradation. 


NUMBER  OF 


TRANSMISSION  TIME  (SEC) 


271,  272,  291,  302,  and  303  seconds,  but  are  again  reflected  in  the  statistics  computed.  .\t  0 
percent  d<'|:;radation,  over  97  percent  of  the  messages  took  less  than  7  seconds.  This  percentage 
decreased  for  15  and  30  percent  degradation  where  only  72.5  percent  and  56  percent  of  all 
initiating  messages  took  less  than  7  seconds,  respectively. 

Intensity  had  no  signiflcant  effect  an  transmission  time.  Between  73  and  77  percent  of  all 
messages  had  a  transmission  time  less  than  7  seconds  for  all  intensity  levels.  In  conclusion,  the 
only  factor  that  had  any  significant  effect  on  transmission  time  was  communication  degradation. 

6.  .Number  of  Missions  Initiated.  Completed  and  Expccir.i  A  completed  fire  mission 
includes  a  call  for  fire,  M  TO,  at  least  one  SHOT  and  an  end-cf  mission  (EO.Vl).  .An  initialed  fire 
mission  by  definition  is  a  call  for  fire  (FR  GRID)  followed  by  a  message-to-observer  (MTO). 
The  number  of  expected  fire  missions  is  the  number  of  fire  missions  in  the  database.  Table  9 
gives  the  number  of  missions  completed,  initiated  and  expected  for  each  intensity  level. 


TABLE  9.  Number  of  Missions  Initiated,  Completed  and  Expected 


INTENSITY 

TOTAL  # 
COMPLETED 
MISSIONS 

TOTAL  # 
INITIATED 
MISSIONS 

TOTAL# 

EXPECTED 

MISSIONS 

LOW 

73 

75 

72 

MEDIUM 

156 

157 

156 

HIGH 

227 

229 

228 

The  total  number  of  initiated  missions  is  larger  than  the  expected  number  of  missions  for 
each  intensity.  This  is  largely  due  to  the  inability  of  the  FIST  HQ  to  distinguish  between 
duplicate  calls  for  fire.  Since  the  FIST  HQ  believed  these  duplicate  fire  requests  to  be  new  calls 
for  fire,  the  messages  were  forwarded  to  the  FDS  or  MFDS  at  the  teams'  discretion.  In  three 
instances,  the  FDS  received  one  copy  of  the  fire  request  while  the  MFDS  received  another.  In 
one  case,  the  FDS  received  two  copies  of  a  FR  (at  different  time  intervals)  from  the  FIST  HQ 
during  a  30  percent  degradation  cell.  This  type  of  oversight  during  degraded  communications 
was  not  unique  to  any  one  FIST  HQ. 

Confusion  caused  by  heavy  message  traBBc  can  be  seen  in  one  particular  high  intensity, 
30-pcrcent  degradation  cell.  During  the  two  hours  and  ten  minutes  of  testing,  two  fire  missions 
were  canceled  because  the  MFDS  was  busy  (MFDS  can  run  only  one  mission  at  a  time;  the  fire 
requests  could  have  been  sent  to  the  FDS),  but  the  FIST  never  re-initiated  the  calls  for  fire  or 
re-routed  the  requests  to  the  FDS.  As  a  result  only  17  out  of  the  19  expected  missions  were 


completeti.  During  a  low  intensity,  0-percent  degradation  cell,  FIST  3  spuriously  sent  a  SA 
(subsrciuent  adjust)  LASER  mission.  This  fire  mission  was  never  completed. 


B.  Anrdvsjs  of  Questionnaires 


The  I'IST  IIQ  were  required  to  fill  out  questionnaires  at  the  end  of  the  FIST  DMD  training 
phase  and  at  the  end  of  the  test.  Mr.  Leonard  Cunningham  and  Major  Grim  of  the  Field 
.Artillery  Board  developed  the  questionnaires  for  the  FIST  Force  Development  Testing  and 
Experimentation  (FDTE).  (See  Appendix  B  for  sample  questionnaires)  Seventeen  of  the  FIST 
IIQ  members  filled  out  the  end-of-training  questionnaire.  Twenty  team  members  filled  out  the 
end-of-test -questionnaire. 


1.  F.nd-of-Training  Questionnaire.  Part  I  of  this  questionnaire  was  designed  to  provide 
information  on  the  FIST  HQ  members'  educational  background  and  previous  military 
experience.  .Ml  of  the  team  members  reported  having  at  least  a  high  school  education.  .All  of 
them  had  previously  worked  as  a  member  of  a  fire  support  team  with  their  experience  ranging 
from  one  month  to  seven  years.  A  majority  of  the  team  members  (H  of  17)  were  FIST 
members  just  before  coming  to  HEL  for  the  test.  A  majority  of  the  team  members  (15  of  17) 
had  also  taken  an  ARTEP  as  a  member  of  a  FIST  HQ  and  rated  their  teams'  .ARTEP 
performance  as  either  excellent  or  superior.  Eleven  of  the  seventeen  team  members  had  received 
their  13F  .MOS  through  the  Army  Institute  of  Technology  (AIT).  Four  team  members  had  been 
trained  through  the  Field  Artillery  Oflicer’s  Basic  Course  (FAOBC).  The  remaining  two 
subjects  had  received  on-the-job-training.  Only  two  of  the  seventeen  team  members  had  prior 
experience  with  a  DMD  and  of  these  two,  one  rated  himself  as  "good”  and  the  other  as 
"outstanding"  at  its  operation.  The  majority  of  the  team  members  (14  of  17)  had  not  operated 
any  T.ACFIRE  devices. 

In  the  second  part  of  the  end-of-training  questionnaire  the  team  members  evaluated  the 
training  session.  The  majority  (14  of  17)  of  them  thought  sufficient  time  had  been  spent  in 
training.  Most  of  the  team  members  (12  of  17)  reported  the  classes  were  well  organited  and  the 
vocabulary  understood.  They  also  reported  the  platform  instructions  were  clear  and  concise  and 
the  objectives  of  the  lessons  clearly  stated  (14  of  17).  There  was,  however,  uncertainty  as  to 
whether  the  situations  presented  in  the  instructions  were  realistic.  Eight  of  the  17  team 
members  said  the  situations  were  realistic.  Whereas  five  of  them  said  they  were  not  realistic  and 
four  team  members  were  uncertain.  The  majority  of  the  team  members  (14  of  17)  reported  that 
the  FIST  DMD  became  easy  to  operate  in  the  time  allotted  for  training.  They  also  said  the 
tasks  in  the  manual  were  explained  clearly  (12  of  17).  The  majority  of  the  team  members  also 
said  they  understood  the  manual  (9  of  15)  and  could  use  its  directions  to  perform  a  task  (14  of 
17). 


In  Part  III  of  this  questionnaire  the  team  members  distinguished  between  those  tasks  they 
h,ad  performed,  those  tasks  they  were  taught  but  had  not  performed,  and  those  tasks  they  were 
not  taught  The  majority  (10  of  17)  of  the  team  members  reported  having  been  taught  all  of 
the  tasks  on  the  list.  Twenty-eight  of  the  42  tasks  listed  in  this  section  were  reported  as  having 
been  performed  by  all  the  team  members. 


In  I’:irt  I\'  of  the  end-of-training  questionnaire,  the  FIST  HQ  members  were  asked  if  they 
had  any  problems  operating  the  FIST  DMD  and,  if  so,  if  they  had  solved  the  problems.  Twelve 
of  the  It)  team  members  reported  having  no  problems  with  the  DMD.  Of  the  four  team 
members  w lio  rejjorted  having  problems  with  the  DMD,  three  said  they  had  solved  the  problem. 
Fourteen  of  the  16  team  members  said  they  could  operate  a  FIST  DMD  under  most 
cirrumst arices.  The  other  two  team  members  did  not  think  they  could  operate  the  DMD  under 
conditions  of  communication  degradation. 

2.  Knd-of-'l~est  Questionnaire.  Part  1  of  the  . ;l-of-test  questionnaire  was  exactly  the  same  as 
Part  II  of  the  eiid-of-training  questionnaire.  The  FIST  HQ  members  were  asked  to  remember 
their  training  session  in  answering  the  questions.  Their  responses  were  basically  the  same  as 
they  had  been  for  this  section  in  the  end-of-training  questionnaire.  The  majority  of  the  team 
members  (16  of  20)  thought  sufficient  time  had  been  spent  in  training.  They  also  reported  the 
platform  instruction  was  clear  and  concise  with  the  vocabulary  understood.  Twelve  of  the 
twenty  team  members  thought  the  situations  in  the  instructions  were  realistic  and  the  tasks  in 
the  manual  explained  clearly.  Four  of  the  team  members  were  uncertain  about  the  realism  of 
the  situations  and  the  clarity  of  the  tasks  in  the  manual.  The  other  four  of  them  thought  the 
situations  were  unrealistic  and  the  tasks  not  explained  clearly.  The  majority  of  the  team 
members  thought  the  classes  were  well  organized  (18  of  20)  and  the  objectives  of  the  lessons 
clear  at  the  beginning  (14  of  20).  Also,  the  majority  of  the  team  members  (18  of  20)  indicated 
the  equipment  was  easy  to  operate  in  the  time  allotted.  Most  of  the  team  members  (16  of  20) 
could  understand  the  manual  and  use  it  to  perform  a  task. 

Part  II  of  the  end-of-test  questionnaire  consisted  of  a  list  of  tasks  which  were  the  same 
tasks  a.s  in  Part  Ill  of  the  end-of-training  questionnaire.  The  team  members  were  asked  to 
check  off  those  tasks  they  had  performed  during  the  test  and  then  to  rate  their  performance  on 
he  task.  They  used  the  following  code  in  rating  their  performance:  1-needed  a  lot  of  help  to 
perform  t:isk:  2-needed  some  help  to  perform  task;  3-nceded  no  help  to  perform  task,  but  was 
slow;  t-p(iformed  task  quickly  with  no  help  and  no  problems. 

Sever. il  of  tlie  tasks  in  this  part  of  the  end-of-test  questionnaire  were  not  performed  by  any 
of  the  team  members.  For  those  tasks  which  the  team  members  did  perform,  the  majority  rated 
themselves  with  a  four  for  each  task.  This  indic.ated  that  the  team  members  believed  they 
performed  the  tasks  quickly  with  no  help  or  problems.  All  of  the  te'->m  members  except  one 
reported  they  could  operate  and  maintain  the  FIST  DMD  under  most  conditions.  Sixteen  of  the 
twenty  tram  members  reported  they  did  not  have  any  problems  operating  and  maintaining  the 
FIST  DMD  as  a  result  of  inadequate  training.  Of  the  other  four  team  members,  one  did  not 
answer  this  question,  two  failed  tc  say  what  problems  they  had  had,  and  one  team  member 
reported  tie  needed  more  field  training. 

VI  CONCLUSIONS 

Soft  w. ire,  intensity,  communication  degradation,  software-intensity  interaction  and 
intensity -degradat ion  interaction  all  had  a  significant  effect  on  message  traffic  through  the  FIST 
HQ.  ,\  change  from  0  to  la-percent  communication  degradation  resulted  in  an  average  increase 
of  .3.'}  percent  in  the  number  of  messages  generated  V  change  from  0  to  30-perrent 


communication  degradation  resulted  in  an  average  increase  of  73  percent  in  the  number  of 
messages  generated.  Medium  intensity  generated  75  percent  more  messages  than  low  intensity 
and  high  intensity  generated  Hi  percent  more  messages  than  low  intensity,  on  the  average. 
Software  was  added  as  a  factor  in  the  experiment  to  control  for  the  variance  induced  by  the 
change  in  software.  Knowing  that  the  change  between  software  versions  was  significant  and  the 
second  set  of  software  was  more  correct  tactically,  only  the  second  half  of  the  test  was  analyzed 
for  the  other  measures. 

Since  the  FIST  DMD  allows  only  four  transmissions  and  then  voice  contact  must  be  made 
to  resynchronize  authenticator  codes,  the  number  of  transmissions  of  a  given  message  before  an 
acknowledgement  is  received  is  important.  Voice  transmissions  on  digital  nets  cause  net 
contention.  In  15  percent  degradation  .3  percent  of  the  messages  required  more  than  four 
transmissions  and  in  30  percent  degradation  6.4  percent  of  the  messages  required  more  than 
four  transmissions.  Although  these  percentages  are  small,  the  total  number  of  messages  is  quite 
large,  and  the  actual  number  of  voice  transmissions  required  may  be  tactically  significant.  It  is 
important  to  also  note  that  fire  direction  net  usage  ranged  from  2.6  to  10.0%  with  only  one  of 
the  three  or  four  FISTs  operating  and  without  the  message  traffic  between  TACFIRE,  BCS,  and 
VFMEDs;  therefore,  net  contention  could  be  a  serious  problem  when  the  net  is  fully  loaded. 

The  median  service  time  for  messages  was  influenced  significantly  by  team,  message  type, 
replication,  intensity,  degradation,  and  many  of  the  interaction  terms.  It  is  not  surprising  that 
when  measuring  a  human  response  time  that  the  humans,  in  the  FIST  HQ,  are  the  most 
significant  factor.  Replication  being  significant  in  this  instance  can  be  translated  to  a  slight 
learning  cllect  since  the  first  replicate  occurred  on  the  first  three  days  of  testing  and  the  second 
replicate  occurred  on  the  last  three  days.  An  increase  of  32  percent  in  median  service  time  for 
fire  requests  and  ATIs  combined  was  observed  from  0  to  30— percent  communications 
degradation  and  an  increase  of  34  percent  was  observed  as  intensity  increased  from  low  to  high. 
The  combined  effect  of  intensity  and  degradation  is  most  noticeable  in  high  intensity.  That  is, 
communication  degradation  has  little  effect  within  low  intensity  or  medium  intensity,  but  has  a 
very  large  effect  in  high  intensity.  Although  fire  requests  take  longer  to  process,  in  general,  than 
ATIs,  as  communication  degradation  increases  within  high  intensity,  the  rate  at  which  service 
time  (which  is  both  the  time  spent  in  the  FIST  DMD  queue  and  the  human  processing  time) 
increases  for  ATIs  is  considerably  higher  than  the  rate  of  increase  for  fire  requests. 
Consequcnlly,  at  30  percent  degradation  ATIs  take  longer  to  process  than  fire  requests.  Service 
time  in  high  intensity  increases  179  percent  for  ATIs  and  46  percent  for  fire  requests.  What  this 
would  indicate  is  a  queueing  problem  at  the  FIST  HQ.  Fire  requests  are  higher  priority  than 
ATIs  and  are  selected  out  of  the  queue  before  ATIs  for  processing.  Therefore,  although  it  may 
not  take  as  long  to  process  ATIs,  they  are  remaining  longer  in  the  FIST  DMD  queue  until 
finally  their  service  time  exceeds  that  of  fire  requests.  Data  also  seems  to  indicate  that  high 
intensity  and  30-percent  communications  degradation  was  the  point  at  which  fire  support 
coordination  operations  began  to  seriously  degrade  (See  Figure  18). 

Many  software  enhancements  were  recommended  for  the  FIST  DMD;  nearly  all  of  them 
have  been  implemented  by  the  developers  and  added  to  the  FIST  DMD  specification.  For 
example,  new  messages  entering  the  input  queue  will  now  be  compared  with  the  others  in  the 
queue  and  discarded  if  they  are  duplicates.  Several  functional  problems  were  also  identified  and 
reported  to  the  developers.  For  example,  the  FIST  DMD  failed  to  reset  the  try  number  to  zero 
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before  forwarding  messages  in  the  automatic  mode.  If  a  message  took  more  than  one  try  to 
reach  the  f'IST  DMD,  the  number  of  tries  available  to  the  FIST  HQ  to  forward  the  response 
message  was  reduced.  This  also  would  disrupt  the  authentication  sequence  used  by  TACFIRE 
and  increase  the  number  of  times  the  FIST  HQ  would  have  to  establish  voice  contact  to 
resynchronizc  authenticator  codes. 

In  summary,  the  feasibility  of  using  the  automated  techniques  of  ACE  and  the  CPX 
Research  Facility  for  performing  fire  support  control  experiments  and  for  training  soldiers  in  the 
operation  of  the  FIST  DMD  was  successfully  demonstrated. 
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APPENDIX  A 

(Contingency  Tnble  Annlyeb) 


Contingency  table  analysis  is  a  method  used  to  make  direct  inferences  about  whether  two 
or  more  population  distributions  are  identical  to  some  theoretical  form.  Ordinarily,  the  reason 
for  comparing  such  distributions  is  to  find  evidence  for  independence  of  attribute  or 
experimental  conditions.  In  short,  we  employed  a  test  for  independence  for  each  experimental 
unit  in  our  design  matrix. 


The  general  procedure  is  to  statistically  compare  the  sample  or  observed  frequency  for  each 
experimental  unit  to  the  theoretical  expected  frequency. 

The  statistic  used  to  test  if  the  observed  frequency  for  each  treatment  combination  is  equal 
to  the  expected  frequency  is  the  chi-square  statistic.  This  statistic  is  defined  as 


.=1  ei 


where  N  is  the  number  of  experimental  units  and  fj  and  e,  are  the  observed  and  expected  cell 
frequencies,  respectively.  The  calculated  statistic  is  then  compared  to  a  tabulated  value  which  is 
based  on  an  alpha  level  equal  to  .05  and  the  number  of  degrees  of  freedom  associated  with  the 
analysis.  The  number  of  degrees  of  freedom  is  equal  to  the  number  of  experimental  unit  minus 
one,  minus  the  number  of  parameters  estimated  from  the  sample  data  which  are  needed  to 
determine  the  expected  frequency.  If  the  calculated  chi-square  statistic  is  larger  than  the 
tabulated  value,  the  hypothesis  that  the  experimental  treatments  are  not  associated  with  the 
MOP  being  analyzed  is  rejected.  One  restriction  is  that  the  sample  size  must  be  sufficiently 
large  so  that  none  of  the  theoretical  frequencies  are  less  than  1  and  not  more  than  20  percent 
are  less  than  5. 

For  MOP2,  which  is  the  frequency  count  of  the  number  of  times  a  message  is  sent  before  it 
is  acknowledged,  the  theoretical  distribution  could  be  determined  for  each  treatment 
combination  without  any  sample  results.  At  zero  percent  degradation,  the  probability  of  having 
a  try  number  greater  than  zero,  which  can  be  interpreted  as  the  probability  of  a  message  not 
getting  through  and/or  an  acknowledgement  not  being  returned  on  the  first  try,  is  zero.  .\t 
fifteen  percent  degradation,  the  probability  of  a  message  getting  through  and  an 
acknowledgement  returned  on  the  first  try  is  recorded  for  each  two-hour  block  run  with 
degradation  to  have  a  try  number  of  zero.  Similarly,  with  30%  degradation,  one  would  expect 
49  percent  of  the  total  messages  recorded  per  two  hour  block  to  have  been  tried  only  once.  The 
theoretical  probability  by  try  is  given  in  Tables  A-1  and  A-2. 

It  was  originally  decided  that  three  separate  contingency  table  analyses  would  be 
performed  for  each  level  of  communication  degradation.  However,  at  zero  percent  degradation 
all  of  the  messages  should  have  been  acknowledged  after  the  first  try.  The  expected  number  of 
messages  by  try  number  for  the  24  cells  run  at  each  level  of  communication  degradation  is 
presented  in  Table  A-3.  It  is  worth  noting  that  since  no  parameter  estimation  was  needed  to 
determine  these  theoretical  distributions,  the  degrees  of  freedom  for  each  analysis  was  equal  to 
the  number  of  cells  minus  one.  These  theoretical  frequencies  were  compared  to  the  observed 
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TABLE  A-1.  Probability  of  a  Message  Being 
Acknowledged  by  Try 
(15%  Degradation) 


Degradation 

Try 

greater 

12  3  3 

15  % 

.723  .201  .056  .021 

TABLE  A-2.  Probability  of  a  Message  Being 


Acknowledged  by  Try 
(30%  Degradation) 


Degradation 

Try 

greater 

12  3  3 

30  % 

.490  .250  .128  .132 

frequencies  using  the  above  described  procedure.  Then,  using  the  contingency  table  analysis 
outlined  above,  can  determined  if  the  other  experimental  factors  had  an  effect  on  the  number  of 
tries  it  takes  before  a  message  is  acknowledged. 

The  first  step  of  this  analysis  was  to  verify  that  the  uniform  random  number  generator  did 
produce  fifteen  and  thirty  percent  total  message  loss  for  each  set  of  twelve  cells  run  under  the 
modified  software.  Using  the  chi-square  statistic  defined  above,  one  could  test  if  in  fEict  the 
observed  and  expected  number  of  messages  never  degraded  under  15  and  30  percent  degradation 
were  statistically  the  same. 

For  fifteen  percent  degradation,  the  chi-square  statistic  was  calculated  as  2.257  with  11 
degrees  cf  freedom  and  found  not  significant  at  alpha  equal  to  .05.  Similarly,  at  30  percent 
degradation,  the  statistic  was  calculated  as  1.175  with  11  degrees  of  freedom  and  again  found 
not  to  be  significant.  In  fact,  over  each  set  of  twelve  cells,  it  was  calculated  that  .8525  and 
.7054  of  the  messages  were  never  degraded  for  15  and  30  percent  degradation,  respectively. 

Having  verified  that  Ether  was  producing  the  desired  degradation  levels  in  our 
communication  network,  the  next  step  was  to  determine  if  intensity  and  team  variability  had  an 
effect  on  the  distribution  of  message  tries  for  acknowledged  messages  at  each  degradation  level. 

At  0  percent  degradation,  one  would  expect  all  of  the  messages  to  be  acknowledged  on  the 
first  try.  As  seen  from  Table  A-3  below,  almost  all  (98.0%)  of  the  messages  had  successfully 
been  sent  and  acknowledged.  It  is  obvious  that  intensity  and  team  variability  had  no  effect  on  a 
message  reaching  its  destination  at  zero  percent  degradation.  The  1.4  percent  of  the  messages 
that  did  not  get  through  on  the  first  try  could  be  attributed  to  Bit  Box  collisions  which  is  a 


TABLE  A*3.  Observed  Number  of  Messages  Acknowledged 
by  Try  (00%  Degradation) 


Try 

Rep 

Software 

Team 

Intensity 

T 

1 

2 

3 

L 

9 

1 

3 

M 

3 

0 

1 

2 

H 

0 

0 

L 

i 

1 

0 

4 

M 

3 

0 

H 

3 

0 

L 

112 

2 

0 

3 

M 

102 

4 

0 

2 

H 

209 

2 

0 

1 

L 

2 

0 

4 

M 

2 

0 

H 

2 

0 

hardware  phenomena.  This  phenomena  occurs  when  two  messages  enter  the  Bit  Box  on 
opposite  ends  simultaneously,  collide  and  then  are  lost. 

A  contingency  table  analysis  was  performed  on  the  12  two-hour  cells  run  at  15  percent 
communication  degradation.  The  observed  number  of  messages  acknowledged  for  try  one,  two, 
three  and  tries  greater  than  three  was  compared  to  the  expected  number.  The  calculated  chi- 
square  statistic  was  44.2  with  47  degrees  of  freedom.  This  statistic  was  not  statistically 
significant  and  one  could  only  conclude  that  the  observed  and  theoretical  distributions  were  the 
same. 

For  30  percent  degradation  the  contingency  table  analysis  again  revealed  that  intensity, 
team  variability  and  replication  did  not  influence  the  number  of  tries  it  took  for  a  message  to  be 
acknowledged.  The  chi-square  statistic  was  30.29  with  59  degrees  of  freedom. 
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Id  conclusion,  based  on  our  experiment,  we  demonstrated  that  the  number  of  tries  it  takes 
before  a  message  is  acknowledged  is  a  function  of  the  percent  degradation  exhibited  in  the 
communications  network  and  it  is  not  statistically  influenced  by  intensity,  team  variability  or 
replication.  The  theoretical  and  actual  frequency  distributions  by  try  number  are  given  in 
Tables  A-  l  through  A-7  for  15  and  30  percent  communication  degradation,  respectively. 


TABLE  A-4.  Observed  Number  of  Messages 
Acknowledged  by  Try 
(15%  Degradation) 


Try 


greater 


Team 

Intensity 

1 

2 

3 

3 

L 

96 

26.7 

7.5 

2.7 

3 

M 

165.5 

46.0 

12.8 

4.7 

H 

191.5 

53.3 

14.8 

5.4 

L 

86.7 

6.7 

2.5 

4 

M 

159.7 

IBI 

4.5 

H 

205.2 

57.1 

m 

5.8 

L 

86 

24 

2.4 

3 

M 

146.6 

40.1 

4.2 

H 

208.8 

58.1 

5.9 

L 

102.6 

28.5 

7.9 

2.9 

4 

M 

153.9 

42.8 

Ba 

4.4 

H 

205.2 

57.1 

1 

5.8 

TABLE  A-6.  Observed  Number  of  Messages 
Acknowledged  by  Try 
(30%  Degradation) 


TABLE  A-7.  Expected  Number  of  Messages 
Acknowledged  by  Try 
(30  %  DEGRADATION) 


L.  4 


► 


I 
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APPENDIX  B 

FIST  DMD  OPERATOR'S  TRAINING 
QUESTIONNAIRE 
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APPENDIX  B 


FIST  DMD  OPERATOR'S  TRAINING 
QUESTIONNAIRE 


NAVE  _ _ _  rank 

UNIT  _  SSAN 

MOS  _ _  NUMBER  OF  MONTHS  IN  MOS 


Circle  the  highest  grade  you  completed  in  school. 

1  2  3  5  6  7  8  9  10  11  12  GED  13  I'l  15  16  OVER  16 


The  purpose  of  this  questionnaire  is  to  gather  data  about  your  operator's 
training  on  the  FIST  DMD.  These  questions  ask  you  about  the  FIST  DMD  training 
and  the  training  materials.  They  also  ask  about  tasks  you  learned  during  your 
school  training.  Answers  will  be  used  to  determine  if  the  training  should  be 
changed  to  obtain  better  results  in  the  field.  After  discussing  each  task, 
you  have  the  opportunity  to  comment  on  any  phase  of  the  training. 


Your  observations  will  be  used  in  making  the  training  evaluation. 
REMEMBER ! !  GIVE  YOUR  HONEST  OPINION 
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The  following  data  will  be  used  for  statistical  purposes  only.  Please  answer 
each  question  as  accurately  as  possible. 

1.  a.  Have  you  ever  worked  as  a  member  of  a  fire  support  team? 

Yes _  No _ 

b.  If  yes,  how  long?  Years _  Months _ 

2.  Were  you  assigned  to  a  fire  support  team  just  before  you  came  here? 

Yes _  No _ 

3.  a.  Have  you  ever  taken  an  ARTEP  as  a  member  of  a  fire  support  team? 

Yes _  No _ 

b.  If  yes,  how  well  did  the  fire  support  team  do? 

Superior _  Excellent _  Good _  Fair _  Poor _ 

iJ.  Did  you  get  your  MOS,  13F,  through  AIT  or  on-the-job  training? 

AIT _  OJT _  FAOBC _  No  Ans _ 

5.  a.  Have  you  had  experience  with  a  digital  message  device  before  you  came 

here  for  the  test? 

Yes _  No _ 

b.  If  yes,  how  well  could  you  operate  the  DMD? 

Outstanding _  Excellent _  Good _  Fair _  Poor _ 

6.  Place  an  X  by  the  TACFIRE  devices  you  have  operated. 


a.  TACFIRE 


PART  Tl 


* 


Place  an  X  in  the  column  which  best  describes  your  opinion. 


Strongly 

Agree  Agree  Uncertain 


Disagree 


Stron 

Disag 


1.  There  was  too  little  time 
allowed  for  the  instruction. 


2.  There  was  too  much  time  al  owed 
for  the  instruction. 


3.  The  platform  instruction  was 
clear  and  concise. 


4.  The  platform  instruction  was 
confusing . 


5.  The  objective  of  each  lesson 
was  clear  from  the  begini.ing. 

6.  I  understood  all  of  the  words 
used  in  the  training. 

7.  The  time  allowed  for  training 
was  just  about  right. 

8.  The  instructions  given  by  the 
instructors  were  confusing. 

9.  The  equipment  is  easy  to 
operate . 


1C.  The  equipment  is  too  complex 
to  learn  in  the  time  allowed. 


11.  The  situations  presented  in 
the  instructions  were  realistic. 


12.  The  classes  were  well 
organized . 


13.  With  the  instructions  I  have 
received,  I  can  now  operate  the  FIST 
DM!  . 


1^.  I  can  perform  a  task  by  follow¬ 
ing  exactly  the  directions  given  in 
thf  operator's  manual. 


15.  The  tasks  in  the  manual  are 
ex[ lained  clearly. 


16.  I  have  no  problem  underst,  nd- 
ing  the  manual . 


7?i 


PART  III 


Indicate  by  a  check  mark  which  of  the  following  operator  procedures,  or  tasks, 
you  have  performed.  Indicate  those  that  were  taught,  but  you  did  not  perform, 
and  indicate  those  tasks  which  were  not  taught. 


Taught,  but  Not 
Performed  Not  Performed  Taught 

1.  Perform  initial  checks: 

a.  Initial  adjustment.  _  _  _ 

b.  Equipment.  _  _  _ 

c.  Power.  _  _  _ 

d.  Communications  and  G/VL:,D 

interface.  _  _  _ 

e.  Transmission.  _  _  _ 

f.  Addresses.  _  _  _ 

g.  Authentication  code  list  book.  _  _  _ 

2.  Installation  of  internal  battery 

pack.  _  _  _ 

3.  Removal  of  internal  battery  pack.  _  _  _ 

4.  External  power  connections: 

a.  Vehicle  battery.  _  _  _ 

b.  Vehicle  radio  mount.  _  _  _ 

c.  External  battery  pack.  _  _  _ 

5.  FIST  DMD  Interface  connections: 

a.  FM  radio  sets.  _  _  _ 

b.  AM/SSB  radio  sets.  _  _  _ 

c.  Crypto  equipment.  _  _  _ 

d.  Radio  remote  equipment.  _  _  _ 

e.  G/VLLD,  _  _  _ 

f.  Wire.  _  _  _ 

6.  Operational  checks.  _  _ 
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Taught ,  but  Not 
Performed  Not  Performed  Taught 


7. 


8. 


9. 

10. 
11. 


12. 


13. 


1U. 


15. 


16. 


Enter  the  following  data  into 
the  FIST  DMD: 

a.  Local  address. 

b.  FIST  DMD  location. 

c.  Net  status. 

d.  Net  assignment. 

e.  Subscriber  addresses. 

f.  Observers  numbers. 

g.  Subscriber  modes. 

h.  Observer  locations. 

i.  Authenticator  codes. 

The  following  message  formats: 

a.  Standard  fire  request. 

b.  Adjustments. 

c.  Registrations. 

d.  Intelligence. 

e.  Information. 

Mission  buffers. 

Message  files. 

Message  transmission: 

a.  Locally  compounded. 

b.  Messages  from  received 
message  queue. 

Message  reception. 

Message  copies  file. 

Mission  data  file. 

Cleaning  the  DMD. 

Troubleshooting. 
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1.  a.  Did  you  have  any  problems  learning  how  to  operate  the  FIST  DMD? 
Yes _  No _ 

b.  If  yes,  what  problems  did  you  have? _ 


c.  Did  you  solve  the  problem? 

Yes _  No _ 

2.  a.  Do  you  feel  that  you  can  operate  a  FIST  DMD  under  most  conditions? 

Yes _  No _ 

b.  If  no,  under  what  conditions  do  you  feel  that  you  can  not  operate  the 
FIST  DMD? 


3.  Use  the  space  below  for  any  comments  you  may  have  concerning  the  FIST  DMD 
or  improving  the  training. 


A'‘PENDIX  C 


FIST  DMD  OPERATOR'S 

END-OF-TEST  QUESTIONNAIRE 

I 

NAME: _  RANK: _ 

^  UNIT: _  SSAN: _ 

u 

The  following  questions  ask  you  at  cut  each  task  you  learned  during  training. 
After  discussing  each  task,  you  will  have  the  opportunity  to  comment  on  any 
part  of  the  training  or  on  any  task. 

Please  be  serious,  work  carefully  and  give  your  honest  answers  about  your 
•  experiences  and  your  feelings. 

Your  observations  will  be  used  in  making  the  training  evaluation  for  this  test. 

REMEMBER!!  GIVE  YOUR  HONEST  OPINION. 

i 

i 

i 
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This  part  of  the  questionnaire  is  exactly  the  same  as  part  II  of  the 
after-training  questionnaire.  These  questions  pertain  to  the  training  only. 
Remember  back  to  the  training  you  had  and  answer  these  questions. 

I 

Place  an  X  in  the  column  which  best  describes  your  opinion. 

Strongly  Strongly 

Agree  Agree  Uncertain  Disagree  Disagree 

I  1.  There  was  too  little  time 

allowed  for  the  Instruction.  _ 


2.  There  was  too  much  time  allowed 
for  the  instruction. 


3.  The  platform  instruction  was 
clear  and  concise. 


4.  The  platform  instruction  was 
confusing . 

5.  The  objective  of  each  lesson 
was  clear  from  the  beginning. 

6.  I  understood  all  of  the  words 
used  in  the  training. 

7.  The  time  allowed  for  training 
was  just  about  right. 

8.  The  instructions  given  by  the 
instructors  were  confusing. 

9.  The  equipment  is  easy  to  operate. 

10.  The  equipment  is  too  complex 
to  learn  in  the  time  allowed. 


11.  The  situations  presented  in 
the  instructions  were  realistic. 


12.  The  classes  were  well 
organized . 

13.  With  the  Instructions  I  have 
received,  I  can  now  operate  the  FIST 
DMD. 


14.  I  can  perform  a  task  by  follow¬ 
ing  exactly  the  directions  given  in 
the  operator's  manual. 


15.  The  tasks  in  the  manual  are 
explained  clearly. 


16.  I  have  no  problem  understand¬ 
ing  the  manual. 


PART  II 


Now  that  you  have  had  field  experience  with  the  FIST  DMD,  you  have  probably 
gained  more  knowledge  In  operating  and  In  maintaining  it. 

Please  place  an  X  in  the  PERFORMED  column  if  you  performed  the  task  in  the 
field  during  the  test.  Under  HOW  WELL  PERFORMED  put  a  number  from  the  code 
below  in  the  space  by  the  X.  If  you  did  not  perform  the  task  in  the  field, 
leave  both  spaces  blank. 

Use  the  following  code  numbers. 

CODE  NO.  HOW  WELL  PERFORMED 

1  Needed  a  lot  of  help  to  perform  task. 

2  Needed  some  help  to  perform  task. 

3  Needed  no  help  to  perform  task,  but  was  slow. 

**  Performed  task  quickly  with  no  help  and  no  problems. 

HOW  WELL 

PERFORMED  PERFORMED 


1.  Performed  the  following  initial  checks: 

a.  Initial  adjustment. 

b.  Equipment. 

c .  Power . 

d.  Communications  and  G/VLLD  interface. 

e.  Transmission. 

f.  Addresses. 

g.  Authentication  code  list  book. 

2.  Installed  internal  battery  pack. 

3.  Removed  internal  battery  pack. 

Made  the  following  External  power  connections: 

a.  Vehicle  battery: 

b.  Vehicle  radio  mount. 

c.  External  battery  pack. 


CODE  NO.  HOW  WELL  PERFORMED 

1  Needed  a  lot  of  help  to  perform  task. 

2  Needed  some  help  to  perform  task, 

3  Needed  no  help  to  perform  task,  but  was  slow. 

y  Performed  task  quickly  with  no  help  and  no  problems. 

HCW  WELL 

PERFORMED  PERFORMED 

5.  Made  the  following  FIST  DMD  interface  connections: 

a.  FM  radio  sets.  _  _ 

b.  AM/SSB  radio  sets.  _  _ 

c.  Crypto  equipment.  _  _ 

d.  Radio  remote  equipment.  _  _ 

e.  G/VLLD.  _ _ 

f .  V/ire.  _  _ 

6.  Made  the  FIST  DMD  Operational  checks.  _  _ 

7.  Entered  the  following  data  into  the  FIST  DMD: 

a.  Local  address.  _  _ 

b.  FIST  DMD  location.  ________  _ 

c.  Net  status.  _  _ 

d.  Net  assignment.  _  _ _ 

e.  Subscriber  addresses.  _  _ _ 

f.  Observers  numbers.  _  _ 

g.  Subscriber  modes.  _  _ 

h.  Observer  locations.  _  _ 

i.  Authenticator  codes.  _  _ 

8.  Used  the  following  message  formats: 

a.  Standard  fire  requests.  _  _ _ 

b.  Adjustments.  _  _ 

c.  Registrations.  _  _ 


d.  Intelligence 


CODE  NO. 

1 
2 

3 

4 

HOW  WELL 

PERFORMED  PERFORMED 


HOW  WELL  PERFORMED 

Needed  a  lot  of  help  to  perform  task. 

Needed  some  help  to  perform  task. 

Needed  no  help  to  perform  task,  but  was  slow. 
Performed  task  quickly  with  no  help  and  no  problems. 


9.  Used  the  mission  buffers.  _  _ 

10.  Used  the  message  files.  _  _ 

11.  Made  the  following  message  transmissions: 

a.  Locally  compounded.  _  _ 

b.  Messages  from  received  message  queue.  _  _ 

12.  Received  FIST  DMD  messages.  _  _ 

13.  Used  the  message  copies  file.  _  _ 

14.  Used  the  mission  data  file.  _  _ 

15.  Cleaned  the  DMD.  _  _ 

16.  Used  troubleshooting  procedures  on  the  FIST 

DMD.  _  _ 

17a.  Do  you  feel  that  you  can  operate  and  maintain  the  FIST  DMD  under  most 
conditions? 

Yes _ .  No _ . 

b.  If  No,  what  is  the  problem?  _ 


18a.  Do  you  have  any  problems  operating  and  maintaining  the  FIST  DMD  because 
of  Inadequate  training?  Yes _ .  No _ . 

b.  If  Yes,  what  are  the  problems? _ 


19.  Use  the  space  below  for  any  comments  you  may  have  on  the  training  you 
received,  on  the  FIST  DMD  itself,  or  on  the  operation  and  maintenance  of  the 
FIST  DMD. 


APrtNDlX  I) 
l.HSSONS  Ll:ARNi;i)  FROM 
TH(i  MST  CrXRF  FXI’FR IMFNT 
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AIM’l'NDIX  D.  LESSONS  LEARNED  FROM  THE  FIST  CPXRF  EXPERIMENT 


Scenarios 

Scenario  definition  is  still  very  subjective.  For  this  experiment  a  scenario  was  defined  to 
be  a  time  ordered  list  of  digital  messages,  and  in  this  case,  messages  that  would  be 
received  by  the  FIST  HQ  from  its  platoon  forward  observers  (FO)  using  TACFIRE  Digi¬ 
tal  Message  Devices  (DMD).  It  was  surprising  that  no  definition  of  "intensity”  could  be 
found  that  was  given  in  terms  of  number  of  fire  missions  (FM)  or  messages  per  hour. 
Hence,  a  "reasonable”  guess  had  to  be  made;  Low  Intensity  -  1  Fire  Mission  per  FO  per 
hour.  Medium  -  2  FM/FO/hour,  and  high  -  3  FM/FO/hour.  The  number  of  .Xrtillery 
Target  Intelligence  (ATI)  messages  was  varied  inversely  from  the  FMs,  and  an  Immediate 
Smoke  mission  was  added  to  each  medium  and  high  intensity  cell  (each  cell  was  2  hours 
long). 

Because  cells  of  the  same  intensity  were  to  be  compared,  several  other  criteria  were 
imposed  on  the  scenario  to  insure  that  task  loading  on  the  FIST  HQ  didn’t  vary 
significantly  between  cells  of  the  same  intensity.  The  ratio  of  Fire  For  Effect  (FFE)  to 
Adjust  Fire  (AF)  missions  was  chosen  as  2:1  (as  per  Ft.  Sill’s  direction),  the  number  of 
adjustments  in  each  AF  mission  was  chosen  as  three,  and  one  fire  mission  in  each  2-hour 
cell  was  designated  as  urgent  rather  than  normal  priority.  After  the  scenario  was 
received,  it  was  realized  that  the  time  interval  between  fire  missions  was  ;dso  a  significant 
factor  that  influences  the  loading  on  the  FIST  HQ.  Since  this  timing  wasn’t  specified  in 
the  scenario  definition,  all  the  fire  mission  time-tags  were  changed  manually  so  that  the 
intervals  between  the  fire  missions  were  the  same  for  each  cell  of  the  same  intensity. 
This  was  a  time  consuming  procedure  (60  man-hours).  Some  factors  that  were  not 
analyzed  in  the  first  CPX  experiment  (but  perhaps  should  have  been)  were  the  ranges  to 
the  t.argets,  the  target  descriptions,  and  how  often  or  which  targets  were  in  range  of  the 
available  fire  support  assets.  These  factors  would  certainly  affect  the  FIST  HQ  percep¬ 
tion  of  the  threat  and  perhaps  the  urgency  attached  to  tasks,  both  of  which  might 
influence  the  results  of  tests  like  this  one. 

The  process  of  verifying  that  the  large,  144-hour  scenario  contained  what  was  requested 
could  only  be  achieved  practicably  through  automatic  means.  The  scenario  was  delivered 
on  computer  tape  and  loaded  into  the  BRL  computer(s)  where  it  could  be  manipulated 
with  the  many  standard  software  packages  included  with  the  UNIX  Operating  System. 
Several  other  programs  were  written  to  examine  the  database  and  display  information 
concerning  the  factors  listed  above.  (Most  of  these  programs  were  written  in  a  con¬ 
venient  pattern  scanning  and  processing  language  named  .AWK.) 

The  messages  in  the  scenario  had  to  be  converted  from  a  "pseudo  TACFIRE”  format  to 
the  "Fixed  Format”  used  by  DMDs.  The  program  written  to  do  this  was  "table  drivenr 
This  type  of  program  is  relatively  quick  to  write  and  fast  in  execution,  however,  it  is 
intolerant  to  errors;  therefore,  any  deviation  from  the  expected  input  format  produced  an 
error.  There  were  many  format  errors  in  the  input  database  (e.g.,  the  abbreviation  for  a 
target  disposition,  DLSPO,  was  often  missing  the  ”1’.’  DSPO)  that  required  manual  altera¬ 
tions,  an  unexpected  time  consuming  task  on  the  large  database. 


rhc  i’vo  simulator  programs,  FOSCE  (Forward  Observer  SCEnario)  and  FDS/MFDS 
(l  ire  Direction  Simulator/Mortar  FDS)  interactively  simulated  both  tactical  equipment 
and  its  human  operators.  FOSCE  mimicked  the  actions  of  the  platoon  forward  observers 
that  work  for  the  FIST  HQ  while  the  FDS  programs  simulated  generic  artillery  battalion 
and  mortar  fire  direction  centers,  respectively,  executing  fire  missions.  It  had  to  be  deter- 
miiKcl  exactly  how  the  simulators  should  react  to  the  many  different  events  that  could 
occur  during  the  scenario.  This  required  group  participation  and  each  possible  event  had 
to  be  discussed.  Many  different  factors  were  taken  into  consideration  such  as  tactical 
realism,  physical  constraints,  and  the  test  design.  Most  of  the  events  could  be  handled  in 
more  than  one  way  and  a  decision  had  to  be  made  (that  was  sometimes  arbitrary)  as  to 
how  the  simulator  should  react.  This  problem  was  compounded  by  the  introduction  of 
degraded  communications.  Even  for  a  simple  adjust  fire  mission,  over  30  randomly 
selected  messages  could  be  deleted  for  30%  communications  degradation  level,  and  the 
simulators  had  to  adjust  to  react  to  this  (note  Figure  5). 

Some  operations  that  are  normally  handled  via  voice  with  TACFIRE  had  to  be  imple¬ 
mented  using  digital  means  for  the  simulators.  For  example,  fire  missions  from  FOSCE 
could  be  rejected  or  ended  by  sending  a  freetext  message  "MISSION  REJECTED”  If  a 
target  number  had  been  assigned,  FOSCE  would  send  an  End  Of  Mission  and  Surveil¬ 
lance  (ES)  message  and  then  wait  for  the  next  mission.  An  ES  message  would  cause  the 
FDS  or  MFDS  to  end  a  mission  already  in  progress. 

Despite  the  capability  built  into  the  simulators  to  handle  degraded  communications, 
"deadlock"  situations  still  occurred.  This  would  happen  when  a  message  that  was  lost 
rioiii  one  simulator  was  required  by  another  simulator  to  continue  processing.  A  common 
examitlc  of  this  occurred  when  a  Subsequent  Adjust  message  from  FOSCE,  which  was  to 
be  relayed  automatically  to  the  FDS  by  the  FIST  DMD,  was  lost  during  the  relay, 
.Although  the  FIST  DMD  warned  the  operator  when  such  a  situation  existed,  the  warning 
w;is  sometimes  missed,  inadvertently  deleted,  or  more  urgent  problems  forced  a  delay  in 
reacting  to  the  warning  and  it  was  forgotten.  Hence,  FOSCE  was  content  because  the 
message  it  sent  was  acknowledged  by  the  FIST  DMD;  FOSCE  was  waiting  for  a  SHOT 
message.  Hut  the  Subsequent  Adjust  message  for  which  the  FDS  was  waiting  never 
arrived,  both  simulators  were  waiting  for  something  from  the  other.  It  was  noted  by  a 
Field  .Artillery  School  instructor  'hat  this  situation  is  not  uncommon  in  the  real  world. 
To  solve  this  problem,  the  simulators  had  to  be  able  to  respond  to  queries  from  the  FIST 
11(2.  ITettexl  fire  mission  "status”  messages  were  defined  so  that  the  1  1ST  HQ  could 
impure  about  the  current  status  of  a  fire  mission  by  referencing  cither  a  Target  Number 
or  an  Observer  Identification  number  and  DMD  mission  buffer.  These  intjuiries  could  be 
sent  to  either  the  FOSCE  or  FDS  programs,  and  based  upon  the  response,  the  FIST  HQ 
could  lake  action  to  fix  the  deadlock  situation  or  end  the  mission. 

During  the  discussions  concerning  the  reactions  of  the  simulators  to  missing  or  erroneous 
ines.,agrs,  it  became  quite  obvious  that  it  is  easier  to  teach  an  application^-  expert  to  pro¬ 
gram  than  the  reverse. 


A  real  time  display  program,  named  ADIS  (ACE  Display  and  Information  System), 
allowed  the  controllers  to  see  all  the  messages  as  they  passed  between  real  and  simulated 
players.  The  display,  along  with  a  chronological  listing  of  the  scenario,  was  used  by  the 
controllers  to  track  the  progress  in  the  test  cells.  This  proved  to  be  essential  in  identify¬ 
ing  and  collecting  information  on  unusual  events,  which  normally  developed  quickly  and 
were  hard  to  trace.  This  was  even  more  important  during  the  extensive  debugging  phase 
of  the  experiment.  The  Field  .Artillery  School  instructor  also  found  it  helpful  in  following 
the  progress  of  the  students  during  the  training  phase. 

Special  messages  were  developed  to  allow  the  simulators  to  add  information  to  the  data¬ 
base.  These  messages  were  sent  to  reserved  addresses  so  that  they  could  be  easily 
identified  in  the  database  later.  The  simulators  did  not  expect  acknowledgement  for 
these  me.ssages.  Some  of  these  messages  were  displayed  by  ADIS  for  the  controllers  to 
identify  special  events,  such  as  a  simulator  not  receiving  an  acknowledgement  after  four 
transmission  attempts,  or  the  reception  of  a  duplicate  fire  mission  by  the  FDS  (i.e.,  same 
as  one  already  in  progress).  Other  messages  were  simply  entered  into  the  database  for 
use  during  data  reduction.  These  were  typically  messages  that  announced  that  a  particu¬ 
lar  simulator  was  beginning  or  ending  a  fire  mission.  They  identified  the  time  that  the 
simulators  believed  a  fire  mission  started  or  ended  regardless  of  the  actions  of  the  live 
players.  These  messages  were  used  by  the  data  reduction  software  to  identify  when 
events  actually  began  or  ended  despite  the  confusion  caused  by  the  communications 
degradation. 

The  capability  to  start  or  end  a  cell  anywhere  within  that  cell  was  implemented  to  allow 
the  controller  to  restart  a  cell  if  a  computer  hardware  malfunction  was  encountered,  thus 
eliminating  the  need  to  redo  a  complete  cell.  This  capability  was  utilized  only  two  or 
three  times  during  the  entire  experiment  but  it  did  save  several  hours  of  test  time. 

camera  and  microphone  were  placed  in  the  FIST  vehicle  mock-up  to  record  the  face  of 
the  FIST  DMD  and  the  crew’s  conversations.  These  were  simultaneously  recorded  with 
the  .ADIS  Display  so  that  a  complete  picture  could  be  obtained  at  a  later  time  to  identify 
specific  parts  of  the  scenario  that  caused  problems  and  the  events  that  led  up  to  them. 
Wlien  the  ADIS  display  was  combined  with  the  actual  actions  of  the  crew,  it  presented  a 
comprehensive  picture  of  how  the  test  group  (FIST)  interacted  with  the  entire  system. 

Hardware 

The  Hit  Boxes  (MODEMs  that  convert  between  Frequency  Shift  Keyed  (FSK)  TACFIRE 
signals  and  standard  RS-232  signals)  do  not  prevent  collisions  from  occurring:  therefore, 
even  "perfect”  communications,  wasn't.  The  FIST  DMD  contained  a  "net  monitoring” 
function  to  reduce  message  collisions.  This  feature  was  not  implemented  in  the  Bit 
Boxes,  and  it  was  originally  believed  that  the  lack  of  this  was  the  reason  for  most  of  the 
collisions;  however,  this  was  not  the  case.  Of  the  4764  messages  that  were  sent  during 
the  seeond  half  of  the  experiment  under  "perfect"  communications  conditions,  109  had  to 
be  sent  more  than  once.  There  was  only  one  collision  resulting  from  two  original  mes¬ 
sages  being  transmitted  simultaneously;  the  rest  were  collisions  between  a  message  and 
an  acknowledgement  (ACK).  A  look  at  <hese  messages  revealed  that  ‘1  out  of  4  of  them 
were  sent  from  the  FIST  DMD  (the  rest  were  sent  from  the  simulators).  This  exposed  a 


basic  problem,  not  in  the  hardware,  but  in  the  ACE  software.  The  actual  TACFIRE 
hardware  transmits  ACKs  immediately;  however,  those  in  the  experiment  normally  took 
2  to  3.'>  seconds.  This  acknowledgment  delay  was  more  than  enough  to  trigger  the  FIST 
DMO  to  retransmit  the  message.  Hence,  3  out  of  1  of  these  message  collisions  would  not 
ha\e  happened  in  the  "real  world,"  or  only  about  25  of  4700  messages  would  have  actu¬ 
ally  collided  if  the  FOs  were  not  listening  to  the  net  before  transmitting.  Whether  the 
delay  in  the  ACK  was  caused  by  the  ACE  Ether  program  or  the  simulators  is  not  known; 
however,  a  new  Ether  program  is  being  developed  that  will  solve  this  problem,  and  the 
simulators  can  be  modified  to  send  acknowledgements  faster. 


E.  Data  Reduction 

Rcflucing  the  45,000  messages  recorded  in  this  144-hour  experiment  into  meaningful 
information  was  not  a  simple  task.  The  reaction  of  the  FIST  DMD  operators  to 
degraded  communications  often  produced  results  that  were  unexpected  or  difficult  to 
trace.  .-Vs  mentioned  earlier,  the  slight  delay  in  acknowledgement  messages  (ACK)  cou¬ 
pled  with  the  heavy  message  traffic  often  made  it  difficult  to  match  ACK's  to  messages 
after  the  fact. 

lliiiiian  analysis  was  still  reciuired  to  retrieve  many  results.  Many  of  the  standard  pro¬ 
grams  available  with  the  Unix  Operating  System  were  invaluable  to  assist  in  this  work; 
however,  a  great  deal  of  knowledge  about  the  fire  support  (and  TACFIRE)  system  is 
recpiired  to  trace  and  identify  many  problems.  Much  smarter  data  reduction  programs 
will  have  to  be  developed,  probably  using  artificial  intelligence  techniques,  in  order  to 
automate  the  more  complex  data  reduction  problems. 

F.  I'lST  Operations 

Mi-sing  an  ACK  is  more  detrimental  than  missing  a  message.  For  example,  suppose  an 
acknowledgement  (ACK)  is  sent  from  player  B  in  response  to  a  message  received  from 
player  .X.  If  that  ACK  is  not  received,  then  first,  player  A  believes  that  his  message  was 
not  received  (he  received  no  .'XCK),  and  second,  player  B  thinks  everything  is  going  well 
since  he  received  a  message  and  has  no  way  of  knowing  that  his  ACK  did  not  get 
through.  Consequently,  player  B  continues  processing  as  usual,  but  player  .A  keeps  trying 
to  get  the  message  to  player  .\.  clogging  the  radio  net,  and  filling  player  B's  message 
qui  ue  full  of  duplicate  messages  If  this  happ'ms  to  many  people  in  the  system,  the  extra 
loading  can  be  significant,  especially  if  player  B  happens  to  be  the  one  with  whom  every¬ 
one  else  wants  to  communicate,  (e  g.,  TACFIRE). 

The  FIST  could  continue  fire  sujiport  coordination  operations  through  30  percent  com- 
nuinications  degradation  once  trained  to  do  so.  .\  key  lesson  they  learned  was  to  use  a 
"wait  and  see"  technique  after  failing  to  get  an  ,\CK  to  a  message  after  four  tries.  Usu¬ 
ally  it  is  the  ,ACK.  rather  than  the  message,  that  is  lost  (This  situation  could  become 
(juiie  common  when  a  mobile  observer  with  a  low  power  radio  is  roinmunicating  with  a 
station  th.it  has  a  good  antenna  and  a  hig'a  powered  radio,  for  example,  an  Fi)('.)  Hence, 
by  wailing  a  few  minutes  the  expected  res|)on-e  message  w  ,is  rerei\ed  even  though  the 
.\<  K  never  was.  The  FIST  ,ilso  learned  to  use  the  "status"  messages  to  find  and  fix 
deadlock  sit  nations  thus  resorting  to  digital,  rather  than  voice,  mean-  to  correct  prob¬ 
lems.  However,  they  still  depended  upon  paper  and  pencil  to  keep  trick  of  what  target 


numbers  were  active  along  with  the  progress  of  each  mission. 


u 


Many  software  enhancements  were  rccuinmended  for  the  FIST  DMD;  nearly  all  of  thi 
have  been  implemented  the  developers  and  added  to  the  FIST  DMD  specificati 
(with  its  reference  number): 

3.2.1.19  -  Built-in  relay  mode  can  be  disabled. 

3.2.1.22  -  Multiple  addressing  capability  (as  in  HELBAT-8  FIST  DMD  has  been  added). 

3.2.1.25  -  Transmission  Repeat  Number  (TRN)  is  reset  to  lero  when  forwarding  a  m 
sage  through  the  FIST  DMD  and  when  the  FIST  DMD  authenticators  are  used. 

3.2.1.20.3  -  Duplicate  messages  in  the  message  queue  are  eliminated. 

3.2.1.20.3.5  -  A  special  message  has  been  added  to  alert  the  FIST  DMD  operator  whet 
message  is  received  from  an  unknown  source. 

3.2.1.34  -  ’’Rounds  Complete”  message  (FO  Command)  is  sent  to  the  forward  obsen 
via  a  free  text  message  (until  the  current  DMD  is  upgraded  to  EPROMS  thus  becomi 
reprogrammable) . 

3.2.1.37  •  Mission  denial  feature  has  been  added  to  the  Message  To  Observer  (MTO). 
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NOMENCLATURE 


ACE 

Artillery  Control  Environment 

ACK 

Acknowledgement  (message) 

ADIS 

ACE  display  and  Information  System 

ADP 

Automatic  Data  Processing 

ATI 

Artillery  Target  Intelligence 

EBP 

Bit  Box  Interface  Program 

DCS 

Battery  Computer  System 

BCU 

Battery  Computer  Unit 

BDU 

Battery  Display  Unit 

bn 

battalion 

buy 

battery 

Command,  Control  and  Communication 

CH 

Chief 

Cnid 

Command 

CO 

Company 

CPU 

Central  Processing  Unit 

Cl'XRX 

Command  Post  Exercise  Research  Facility 

CUT 

Cathode  Ray  Tube 

DARCOM 

US  Army  Materiel  Development  &  Readiness  Command 

DNID 

Digital  Message  Device 

EOM 

End  of  Mission  (Message) 

EOT 

End  of  Transmission 

ES 

End  of  Mission  and  Surveillance  (Message) 

EW 

Electronic  Warfare 

ETHER 

Intra-computer  Communications  Network  Software 

FDC 

Fire  direction  Center 

FDS 

Fire  Direction  Simulator 

FFE 

Fire  for  Effect  (Mission) 

FM 

Fire  Mission 

FIST 

Fire  Support  Team 

FISTV 

FIST  Vehicle 

FO 

Forward  Observer 

FOSCE 

Forw  ard  Observer  Scenario  Program 

FR 

Fire  Request 

FR  GRID 

Call  to  Fire  using  Grid  Coordinates  for  Target  Location 

FSE 

Fire  Support  Element 

FSK 

f'rcquency  Shift  Keying 

GDU 

Gun  Display  Unit 

hi:lbat 

Human  Engineering  Laboratory  Battalion  Artillery  Test 

HQ 

Headquarters 

htb 

Howitzer  Test  Bed 

lt 

Lieutenant 

MOP 

Measure  of  Performance 

MFDS 

Mortor  Fire  Direction  Simulator 

MSG 

Message 

MTO 

Message  to  Observer  (Message) 

RDTiE 

sa 

Research,  Development,  Testing  and  Evaluation 
Subsequent  Adjust 

SCORES 

Scenario  Oriented  Recurring  Evaluation  System 

TACITR1-: 

Taetieal  Fire  Direction  System 

TRADOC 

I'S  Army  Training  and  Doctrine  Command 
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